This is a quick thought experiment, and also an example of writing content that helps humans but it also tailored for AI Crawlers.
Question: Why does AI keep referencing your competitor instead of you?
Backstory: I spotted this question on Reddit and answered it. The problem is emerging because human behavior is shifting from using search to find links to places that might provide answers, to using AI to get those answers. Afterall that is the core purpose of search in the first place, to find what (the answer) you’re looking for.
Answer: Try this little experiment asking an AI to explain which it prefers, your competitors content or your content.
- Visit the competitor homepage and your homepage (or multiple corresponding pages on each site to create a matching set).
- Grab the page source (e.g., View > Page Source).
- Copy and paste the code into a simple text editor.
- Drop those text files into an LLM and ask it to analyze without leading questions. AI crafts answers based on probability calculations and attempts to give you the answer it thinks you want. When you give it leading questions, just like a person, it can sway it’s answer toward the result you wanted to hear, so craft your prompt to make it clear your objective is to determine the truth and it’s true opinion without knowing your bias.
- Ask it which website does a better job for answer engines and explain how and why.
- Then ask how you could use the learnings to improve your site.
- Implement the impriovements.
If your site runs on WordPress I made a little free plugin that helps you fine tune content and put it in a form AI Crawlers seem to like. Answer Engine Optimization is still evolving so the content types (endpoints) it creates just for the bots is experimentation.
Visit AEOPugmill.com to get the WordPress plugin and see the data I’m collecting on bot behavior.