Battle of the LLM Giants: Google PaLM 2 vs OpenAI GPT-3.5
Google released PaLM 2 on May 10th, 2023, as a worthy response to OpenAI's GPT-4. At their recent I/O event, Google unveiled the captivating PaLM 2 model family, ranging from the smallest to the largest: Gecko, Otter, Bison, and Unicorn. Not only PaLM2 is better, faster, and smaller than the previous PaLM, but it also outshines gpt-4 in certain areas of reasoning, according to Google PaLM 2 Technical Report ( (see table 5 and table 7).
Like many others, at Outside, we are on the learning journey to adopt LLMs to better serve our outdoor community. Recently, we had the opportunity to put PaLM2 and GPT-3.5 to the test using real-life use cases from Outside. If you are contemplating the choice between Google and OpenAI as your LLM provider, or you simply want to learn how to build a Langchain agent equipped with search and questions-answering from knowledge base tools, I hope this post may offer some inspiration for devising an evaluation framework suited to your domain.
In this post, I'll share our exploration of four key areas:
- Methodology and tech deck: Pinecone, Langchain, LLMs (PaLM2 and GPT-3.5)
- Inference Speed and Answer Quality: comparing performance in Langchain's Retrieval QA chain and Conversation Retrieval chain with code examples
- Agent utilizing tools and following instructions: using Langchain's
conversational-react-descriptionagent with Google search API (SerpApi) - Performance in Small talks and safety questions
Side note: the magic spell I used to prompt midjourney to create the feature image is:
yellowstone park with rainbow background, vintage travel poster style, impressive landscape, impressive panoramas, - ar 16:9 - v 5
Outside embrace the LGBTQ+ community, may your pride month be as colorful, unique, and equally appreciated as rainbow and nature.
