Why Not Just Ask ChatGPT to Analyze a Stock?

It is a fair question, and one we hear often. If a general-purpose chatbot can summarize a novel, draft a contract, and write working code, why not just ask it to analyze a stock? The answer is not that large language models are bad - they are extraordinary at the things they were built for. The answer is that equity research has a specific, unforgiving requirement that general chatbots are not designed to meet, and the gap between "sounds right" and "is right" is exactly where capital gets lost.

This post walks through the failure modes that show up when you treat a general LLM as an analyst, and then describes what a research workflow has to do differently to be worth trusting on a real decision.

The Hallucination Problem Is Worse for Numbers#

The most discussed failure mode of general LLMs is hallucination - confident output that is not anchored to anything real. In a casual context, a hallucinated fact is annoying. In equity research, it is disqualifying.

A model that invents a plausible-looking revenue figure, a margin trend, or a quote from an earnings call has not made a small mistake. It has produced a number that a reader will act on, and the number is wrong. The wording around it will be fluent and confident, because fluency is what these models optimize for. There is no built-in signal that says "this digit is made up." The reader has to know, independently, what the right answer is - which defeats the point of asking.

The problem is structurally worse for numbers than for prose. A paragraph that drifts from the source still reads coherently and a careful reader can spot the drift. A single wrong digit in a margin or a growth rate slides past unnoticed and lands in a position.

This is not a flaw that can be prompted away. It is a property of how general LLMs generate text - token by token, weighted by what is likely to come next, with no requirement that the output match a specific document. Asking the model to "be careful with the numbers" does not change the underlying mechanism.

Training-Data Cutoffs Quietly Make Output Stale#

The second failure mode is less discussed and just as dangerous. A general LLM's knowledge of a company is bounded by its training data, which was assembled at some point in the past and frozen.

That means when you ask a chatbot about a company, you may be reading a description of the business as it existed eighteen months ago. The CEO it names may have left. The segment it describes may have been divested. The guidance it cites may have been retracted on the next call. None of this comes with a warning label - the output reads as current because the model writes in the present tense.

Equity research is, almost by definition, time-sensitive. A description of management strategy from before the most recent earnings call is a description of a strategy that may no longer exist. A model that cannot tell you what it does not know, and cannot reach for the latest filing, is structurally unsuited to the work - even if the language it produces sounds authoritative.

The riskiest version is when the model has partial information about a recent event. It will fill in the gaps with what is likely, not what is true, and the result will read as though it were reporting from the latest disclosure.

No Anchoring to SEC Filings or Transcripts#

The third issue is the most fundamental. A general chatbot is not connected to the documents that matter. When it talks about a company, it is drawing on a statistical summary of everything it has read - news articles, blog posts, forum threads, prior filings - without distinguishing the primary source from the commentary about the primary source.

That distinction is the entire foundation of equity research. The 10-K is the primary source. The earnings-call transcript is the primary source. The proxy statement is the primary source. Everything else is commentary, and commentary is where errors get introduced, repeated, and hardened into "common knowledge."

A general LLM cannot tell you which of its outputs trace back to a filing and which trace back to a recap of a recap of a filing. It does not retrieve the document and quote from it; it produces text that resembles what such a document would say. Sometimes the resemblance is exact. Sometimes it is close. Sometimes it is wrong in a way that no reader could catch without pulling up the actual filing - at which point the reader is doing the research themselves.

This is the line between research and commentary discussed in our piece on what an equity research report is. A claim that does not point to a primary source is not a research claim. It is a plausible-sounding sentence.

What Source-Grounded Research Has to Do Differently#

The alternative is not "a smarter chatbot." It is a different kind of system, built from the ground up around the constraints of equity research rather than the open-ended freedom of general chat.

A source-grounded research workflow has to do several things that a general LLM is not designed to do:

• Pull the actual filing and read from it. Not summarize what filings of this kind tend to say. Not recall what was true in training data. Open the specific 10-K, 10-Q, 8-K, or earnings transcript - free on the SEC's EDGAR database - and work from the words on the page.
• Cite every meaningful claim back to the document it came from. A revenue figure should point to the line item in the filing it was pulled from. A management quote should point to the transcript and the speaker. A claim that cannot be cited is not a claim - it is a guess in a sentence shape.
• Refuse to fill in gaps with what is likely. When the source does not say something, the report has to say so, not generate a fluent-sounding guess. This is the discipline that separates a study from a take.
• Operate against the most recent disclosures. A research report on a company that ignores the latest quarter is not just incomplete; it is misleading. The pipeline has to know what is current, and act accordingly.
• Apply domain knowledge in interpretation, not in invention. Knowing what a deferred-revenue movement implies, or how to read a working-capital swing, is real expertise. Inventing the underlying numbers is not. The two have to stay separate.

The point of these constraints is not to make the system feel safer. It is to make the output verifiable. A reader who disagrees with a conclusion should be able to follow the citation, read the source, and form their own view. That is the contract a research document owes its reader, and it is the contract a general chatbot is not built to honor.

Where General LLMs Genuinely Help#

None of this means general chatbots are useless to an investor. They are very useful - for the right jobs.

A general LLM is excellent at explaining concepts. If you do not know how a debt-to-EBITDA ratio is calculated, or what a "going-concern" qualification means, or how a SPAC structure works, asking a chatbot is faster than digging through a textbook. The answer is general, well-attested, and not time-sensitive - exactly the regime where these models shine.

A general LLM is also genuinely good at summarizing a document you give it. If you paste a 10-K excerpt and ask it to restate the segment results in your own words, the failure modes above largely fall away - the source is in the prompt, the model is constrained to it, and the work is rephrasing rather than retrieval.

What a general chatbot cannot do reliably is be the research process - pull the right filing, read it correctly, cite the specific lines, and refuse to invent the parts it does not know. Treating one as the other is how investors end up acting on confident-sounding output that does not survive contact with the actual document.

If you want to see what filing-grounded analysis looks like end-to-end, our sample reports walk through the structure: claims tied to filings, transcripts as the source for management read, and the parts the source does not support left out rather than guessed.