The Art of LLM Self-Criticism with Google Gemini

Here’s a secret about a Language Learning Model (LLM) like Google Gemini: we can make it even better by teaching it to think critically about its work — to pinpoint potential flaws, spot improvements, and refine its answers. Think of it as giving your super-smart robot helper a critical eye!

Why does this matter? Picture this: Gemini helps you brainstorm ideas for a story… now imagine it can also point out where the plot might get confusing or where a character sounds a bit flat. That’s the power of self-criticism! It’s a game-changer whether you’re a writer, researcher, coder, or just fascinated by how these language models work.

Note: This article spotlights techniques for the Google Gemini Advanced chatbot (a paid service). While these concepts also apply to the free version, we’ll focus on the enhanced capabilities offered by the Advanced subscription.

Why LLMs Need a Dose of Self-Reflection

Let’s be real — even the smartest language models can sometimes produce answers that miss the mark. But what if they could improve their ability to spot those not-so-great responses themselves? That’s where self-criticism comes in.

• What the heck is LLM self-criticism? Simply put, it’s a straightforward process of teaching a language model like Gemini to analyze its work, pinpoint potential flaws, and figure out how to improve. It’s like giving your robot helper a critical eye.
• The power of the redo: Self-criticism is all about iteration — that cycle of trying, tweaking, and trying again. By encouraging Gemini to examine its outputs, we can refine our prompts and guide it toward even more accurate, helpful, and creative responses. This means self-criticism is not a one-time process but a continuous improvement cycle.
• But here’s why it’s REALLY a big deal:
  — Busting those biases: LLMs learn from massive amounts of data, and guess what? That data can be full of hidden biases. For example, if an LLM is trained in news articles, it might learn to associate specific demographics with negative events. Self-criticism helps Gemini question its outputs, highlighting areas where prejudice or stereotypes might slip in.
  — Fact-checking friend: Accuracy is key, especially when using Gemini for research. Training it to analyze its answers helps it spot potential errors, leading to more reliable and trustworthy information.
  — The idea machine: Getting Gemini to critique its work forces it to think outside the box. This translates to a broader range of creative solutions and ideas — no more getting stuck in the same predictable patterns!
  — Harnessing in-context learning: LLMs have a remarkable ability to learn from the examples and information you provide in a conversation. This means that as you interact with Gemini, it’s constantly learning and adapting its responses. Self-criticism taps into this, allowing Gemini to refine its understanding and responses in real-time.

Imagine this: You’re asking Gemini for help outlining a blog post. Not only does it give you some ideas, but it also flags sections that might be too technical or points where the explanation feels incomplete. That kind of self-awareness is pure gold!

Note: The Gemini UI has a feature called “Double-check response”; just click the Google icon to match the content against Google Search data. This is useful for verifying Gemini’s fact-like responses.

Setting the Stage for Self-Criticism

Okay, time to get hands-on! Before we can teach Gemini to be its own best critic, we need to set a few things up:

• Tools of the trade: The most important thing is creating a feedback loop. This means giving Gemini a way to evaluate its work and providing the tools for us to adjust our prompts based on that feedback. It’s also super helpful to keep track of the different prompts you try — think of it as keeping a lab notebook for your LLM experiments.
• Let’s get practical: Say you want Gemini to assess how well it summarizes a news article. Here’s a basic setup.
  — Ask Gemini to summarize the article.
  — Next, immediately follow up with a prompt like, “How accurate is this summary? Does it miss any important points from the original article?”

This simple setup forces Gemini to think critically about its output and to analyze its work.

Prompting for Constructive Criticism

Now for the fun part — we’ll dive into specific prompts designed to make Gemini think critically about its answers. Here are a few powerful techniques:

• The power of “what if?” Contrastive prompts challenge Gemini to consider what might be wrong or missing in its work. Try things like:
  — “What could be inaccurate about this?”
  — “Is there a different perspective to consider?”
  — “What are some potential weaknesses in this response?”
• Step-by-step improvement: Guide Gemini toward a better answer through several prompts. For instance, if it writes a poem that feels a bit basic, you might follow up with:
  — “How could you make this more vivid?”
  — “Could you add a surprising metaphor?”
  — “Can you play with the rhythm or sound of the words?”
• Scenario showdown: This is where we push Gemini’s creativity. Prompt it to generate completely different solutions to a problem. Try asking:
  — “What’s a different way to approach this?”
  — “Can you come up with a more unconventional solution?”
  — “How would you solve this if you had limited resources?”

Let’s illustrate each technique with a scenario:

• The Flaw Finder: You’re working on a research paper, and Gemini summarizes a complex concept. Use a contrastive prompt like, ‘What important details might this summary be missing?’ to make it double-check its work. For instance, if Gemini’s summary of a scientific paper doesn’t mention the key findings, this prompt would help it identify that as a potential flaw.
• The Creative Critic: Gemini’s written you a product description, but it’s boring. Level it up with prompts like ‘How can you make this more engaging?’ or ‘Can you add a touch of humor?’ For instance, if Gemini’s product description is too technical and lacks a human touch, these prompts would help it identify that as a potential flaw.
• Outside the Box: You need ideas for a marketing campaign. Instead of a single solution, prompt Gemini with ‘Give me three wildly different approaches to this campaign.’ This forces it to stretch its creative muscles! For instance, if Gemini’s first two ideas are too similar, this prompt would help it develop a more diverse set of ideas.

Feedback Analysis & Leveling Up Your Prompts

Gemini has done its critical thinking homework; now it’s our turn! It’s time to take the feedback it’s given itself and figure out how to improve our prompts.

• Decoding the critique: Gemini’s feedback might not always be crystal clear. Let’s say it flags a paragraph as “potentially confusing.” Your job is to dig deeper. Is the language too complicated? Does the explanation jump around? Try to pinpoint the why behind Gemini’s self-critique.
• Time to edit: Once you understand where Gemini thinks it went wrong, it’s decision time. How do you change your original prompt to address the issue? Here, you’ll need to use your knowledge of Gemini and sound writing principles (or coding or whatever skill you’re focusing on!).
• The feedback loop: This is the magic part. Take what you learned from Gemini’s analysis, tweak your prompt, and try again! Each time you go through this loop, you should see your prompts strengthen and Gemini’s responses improve.

Example time! Gemini pointed out that its explanation of a scientific concept needed to be more concise. Here’s how your feedback loop might play out:

1. Analyze: Gemini’s flagging that the language itself might be the problem.
2. Edit: Revise your prompt to focus on simplicity: “Explain this concept like I’m a fifth-grader.”
3. Repeat: Feed the new prompt to Gemini and see if the explanation improves!

Mastering the Art: Advanced Tips & Tricks

You’ve got the basics of LLM self-criticism down; now, let’s take it to the next level! Here are a few advanced tips to keep in mind:

• Avoiding the echo chamber: Sometimes, LLMs can get stuck in a rut, especially if you keep feeding them similar feedback. To shake things up, deliberately prompt Gemini for unusual or surprising answers. This helps prevent it from unquestioningly agreeing with its previous assessments.
• Creativity vs. critique: It’s all about balance! You want Gemini to be critical but not so critical it stifles its creativity. If you sense it getting overly nitpicky, switch gears. Prompt it to brainstorm wildly imaginative ideas or focus on generating something entirely off the wall.
• The power of the chain: Prompt chaining takes self-criticism to the next level. Imagine a series of prompts where Gemini first generates a response, then critiques that response then refines it in the next step, and so on. This iterative process can lead to remarkably polished and sophisticated outputs (read my article on prompt chaining).
• The power of roleplay: Think of Gemini as a super-smart actor. Getting it to “play a role” can seriously influence its responses. Try prompting it to be a ruthless critic, a quirky poet, or even a skeptical scientist. By changing Gemini’s “role,” you can unlock different ways for it to analyze its work.

Important note: Feel free to experiment, but always keep track of the changes you make so you can analyze what works and what doesn’t.

The Limits of Self-Criticism

While LLM self-criticism is incredibly powerful, it’s important to remember that it’s not a magic bullet. Sometimes, it might not be the best approach, and human judgment is still crucial. Here’s where to be cautious:

• Understanding context: LLMs can struggle with deeply nuanced situations or interpret subtle social cues. Self-critique won’t fix fundamental misunderstandings, so your human perspective is vital.
• Ethical considerations: Certain topics, particularly those sensitive or controversial, require empathy and careful consideration. While Gemini might help identify problematic language, it shouldn’t replace human judgment in areas with ethical implications.
• Creativity without boundaries: Sometimes you just want Gemini to let loose and brainstorm wild ideas! Too much self-critique can stifle that creative spark. It’s okay to switch gears and focus purely on unfiltered generation.

Key takeaway: Self-criticism is a tool; like any tool, it’s most effective when used strategically. Always consider if it’s the right approach for the task at hand, and remember, there’s no substitute for your critical thinking and awareness!

Conclusion

Alright, you’ve officially leveled up your LLM prompt engineering skills! By teaching Gemini to think critically about its work, you’re unlocking its full potential as a collaborator and creative partner.

Feel free to experiment with the techniques we covered. The more you practice, the better you’ll become at crafting prompts that guide Gemini toward genuinely impressive results.

Check out my reading list of other Google Gemini articles.

This post was created with the help of AI writing tools, carefully reviewed, and polished by the human author.