ChatGPT told me old information like it was current. How do I stop that?
What happened
We asked an AI assistant a simple question: "Are you on the latest version?" of an app we use.
It answered with release notes for version 0.68.1, from a lookup it had done earlier. The app on the screen showed 0.70.2. The answer sounded current, but it was out of date.
Nothing broke, but it would have been easy to plan around the wrong version, chase a bug that was already fixed, or tell someone the wrong thing.
Why the AI did it
The assistant already had an answer in the conversation, so it reused it instead of checking again. Models don't have a strong sense of "this was true an hour ago, but might not be now." Anything earlier in the chat looks like a fact.
Training data has the same problem on a bigger scale. If a model learned about a product months ago, it will describe that version confidently unless it searches.
This matters a lot right now with fast-moving tools. Salesforce, AI connectors and agent platforms change features and names every few weeks. An answer about "what Agentforce can do" from earlier in the quarter may already be wrong.
The fix, step by step
- Ask it to check now. "Look this up again now. Don't use anything from earlier in this chat."
- Make it date every claim. "Start each fact with 'as of [date], per [source]'." If it can't name a source and a date, it's guessing.
- Give it the ground truth when you have it. If your screen says 0.70.2, paste that in. Your app beats its memory.
- For anything you'll act on, open the source yourself. Release notes, the vendor's docs, the settings page. Ten seconds of checking.
- Add a standing instruction. In ChatGPT custom instructions or a Claude project, add: "For versions, prices, features and dates, always re-check and date-stamp. Never reuse an earlier lookup as current."
How to check it worked
- Plant an old answer. Earlier in a chat, give it an outdated fact, like an older version number. Later, ask what's current. It should re-check instead of repeating your old fact.
- Ask about something you know changed recently and see whether it gives a date and a source.
- Spot-check 3 answers against the real source. If any come back undated, tighten the instruction.
Get two new fixes a week
One short email: the two newest entries and one lesson. No spam, unsubscribe any time by replying “stop”.