OCOPD I recently had to build a documentation generation tool that required feeding a 15,000-word codebase summary into an LLM, along with a massive style guide. At first, the model (even with a 128k context window) kept "forgetting" instructions from the beginning of the prompt. After a lot of trial and error, I developed a set of rules to keep mega-prompts stable.
1. The "Sandwich" Structure
LLMs suffer from the "Lost in the Middle" phenomenon. They remember the very beginning and the very end of a prompt, but blur everything in between. I now put the most critical instructions at the top, the massive data dump in the middle, and repeat the critical instructions at the very bottom.
2. Use Indexing for Large Data
When feeding raw data, I always include an index or table of contents. This acts as an anchor.
--- DATA INDEX ---
1. User Authentication Module
2. Payment Gateway Interface
3. Notification Service
--- END INDEX ---
[... massive data follows ...]3. Section Headers are Mandatory
In a 50-word prompt, you can just write paragraphs. In a 10,000-word prompt, you need strict hierarchy. I use uppercase headers wrapped in special characters to make them visually distinct to the tokenizer.
==========
SECTION: STYLE GUIDE
==========
[Content]4. The "Stop and Think" Checkpoint
Before asking the model for the final output, I force it to summarize its understanding of the constraints. This acts as a working memory refresh.
Before generating the final documentation, write a brief bulleted list of the 5 main formatting rules you must follow. Then, generate the document.5. Strip Formatting from Data
When you copy-paste 10k words, you often bring along weird spacing, HTML tags, or markdown artifacts. I run a quick Python script to strip out unnecessary whitespace before sending it to the API. It saves tokens and reduces distraction for the model.
Building massive prompts is less about writing natural language and more about structuring data. Keep it organized, sandwich your instructions, and always force a checkpoint.