Top experts may “hallucinate,” too. 🤯

CompositesAI
1

At the 40th ASC anniversary, Prof. R. Byron Pipes gave a great talk on ASC history—highlighting James Whitney (Tom Whitney’s father), our first ASC president. During his remarks, he showed CompositesAI’s answer to who is James Whitney with a wrong starting date for ASC (Fig. 1). As everyone in the room knows, ASC began in 1985, not the 1970s.

Wrong
Wrong1485×550 60 KB

I did some investigation and traced the error back to:

Stephen W. Tsai (2005), Composites Science and Technology 65:2295–2299, Sec. 2 (Fig. 4).

1-s2.0-S0266353805001843-main_Page_2
1-s2.0-S0266353805001843-main_Page_21653×2206 151 KB

In other words, the “hallucination” started with a leading expert, passed peer review, and then endorsed, published, and propagated by the top journal in our field. To be fair, we all make mistakes like this.

Takeaways:

:small_blue_diamond:Garbage in → garbage out. LLMs mirror their inputs. That’s why CompositesAI is built with experts-in-the-loop and user feedback for continuous correction.
:small_blue_diamond:Not just AI. Print literature can carry forward errors for years—even after errata.
:small_blue_diamond:The good news: Updating AI is fast. Ask CompositesAI now and you’ll get the correct 1985 date (Fig 2).

Correct
Correct1515×671 73.4 KB

Grateful to Prof. Pipes for the great talk and for prompting an interesting story for me to make a post.

2

I think for this kind of facts, we should try web search first. If the result is incorrect, we can add correct ones to the knowledge base.

For example, regarding the founding date of ASC, gpt-5 gives this:

image
image788×126 6.55 KB

The source it uses is a news page from ASC, containing a short video of interviewing Dr. Tsai, which seems done during the SSDM this year.

Regarding James Whitney, we need to be specific when using the general models:

image
image798×460 27.6 KB

I think if we want to use web search in compositesai, we can adjust open webui so that it will add extra keywords like “composites” when searching.

3

Good suggestion. Can we combine website and the documents we uploaded? And let expert information be the main reference. Priority given to expert certified knowledge. Everybody can AI-search the internet. We are promoting expert certified knowledge.

4

Maybe, we should instruct CompositesAI the following

  1. RAG from our expert database first.
  2. If no entries in our database, ask the user whether they would like CompositesAI to search the Internet to find an answer.
  3. If yes, proceed with the web search.