Microsoft has shared the insights it has gleamed after the first week of integrating ChatGPT into its Bing search engine.
Over the past week, the company has been “testing with a select set of people in over 169 countries to get real-world feedback to learn, improve, and make this product what we know it can be”, and feedback on the AI-generated responses have been “mostly positive”, according to Microsoft, with “71 per cent of you giving the AI-powered answers a ‘thumbs up’.”
This is hardly scientific testing, but the feedback is interesting.
Microsoft admits to “finding our share of challenges with answers that need very timely data like live sports scores”, and queries involving “more direct and factual answers such as numbers from financial reports”.
Despite its name, ChatGPT appears to lose the plot during extended talk sessions. Once chats reach 15 questions, “Bing can become repetitive or be prompted/provoked to give responses that are not necessarily helpful or in line with our designed tone.”
Microsoft adds – “very long chat sessions can confuse the model on what questions it is answering.”
It’s interesting Microsoft are releasing such preliminary results; with Google’s recent chat disaster, maybe transparency is the key.
