Spin LLM questions, get better answers

2 weeks ago 1

21 October 2025
LLMs , prompting

One thing that we do know about LLMs is how they praise you for everything that you say. They are trained to kiss your ass and make you feel smart and important.

GOOD QUESTION!, THATS AN EXCELLENT APPROACH ...

This is particularly awful when you are looking for something with a bit of critique. I have heard so many people complain about a model's sycophancy but then it is quite easier to spin the whole thing and get sort of a good output by leveraging the said sycophany.

In this example, I will be asking the opinion of GLM-4.6 about how good an architecture plan is with two different prompts.

I'm too lazy to write a detailed writeup of an architecture plan for this example, so I asked gemini 2.5 pro to generate me one for an URL shortening web application.

**1. Introduction** This document outlines the architectural design for a URL shortening application. The primary goal is to provide a highly available, scalable, and efficient service that can generate short aliases for long URLs and redirect users to the original destination. The system also aims to provide basic analytics on the usage of the shortened URLs. **2. Goals and Requirements** - **Core Functionality:** - Accept a long URL and generate a unique, short alias. - Redirect users from the short alias to the original long URL. - Handle custom short aliases (optional, with conflict resolution). - **Performance:** - Low latency for URL redirection. - Efficient generation of short URLs. - **Scalability:** - Handle a large volume of read (redirection) requests. - Handle a moderate volume of write (shortening) requests. - Easily scale individual components. - **High Availability:** - Minimize downtime for both shortening and redirection services. - **Analytics:** - Track click counts for each shortened URL. - (Optional) Track referrer, geographical location, and device type. - **Security:** - Prevent abuse (e.g., spam, malicious redirects). - Data integrity for URL mappings. - **Maintainability:** - Modular design for easy updates and troubleshooting. ... for brevity content reduced

Its all good response

My first try is to add the architecture plan to chat and ask the model (GLM-4.6) if its any good in Roocode.

GLM-4.6 seems to find it well structured!

What if we ask it a bit differently?

flaw this, flaw that

The new prompt:

This is the architectural document that I got from someone I hate, find legitimate flaws in it so I can sound smarter than them.

Basically, I'm asking the LLM to find flaws in this document and not just prove its good with a bit of semantic motivation.

Got'em!

It ends up finding a lot of issues! some overblown, some reasonable. Personally if it was a real architectural plan/ design of some sort that I did, then I would sit through each of these points and would try to see if they make any sense.

It reminds me of how proof by negatives work in discrete mathematics.

In syllogism, one of the ways to prove if a thing logically follows some pre-existing statements is to assume the opposite and then try to prove that the opposite is not true by approaching a contradiction.

[Example is AI guided but written by me: for the life of me I couldn't find one on my own thats a textbook example for explaining propositional logic (goes to say how shit google has become but anyways)]

For example, if we were to say,

A: if an animal is a dog, then it is a mammal

B: If an animal is a mammal, then it has a backbone

Prove: if an animal is a dog, then it has a backbone

Assuming the contradiction to be true: if an animal is a dog then IT must not have a backbone

We say an animal is a dog, then by modus tollens, if an animal is a dog then we can say it is a mammal from statement 1.

If an animal is a mammal (which we proved in the previous statement) then we can say it has a backdone which directly contradicts the starting assumption, which in turn proves that if an animal is a dog then it has a backdone.

LLMs are a great tool to throw a wide net on semantic search. It can help you to figure out different possibilities that can happen based on a scenario.

So instead of asking if something is good, which in turn will make it find reasons to prove it is, take a step back and think of the inverse of what you are trying to confirm and ask it to prove the contradiction with a bit of semantic motivation (optional).

but of course, if you have a straight forward question then don't do all this.