If you are starting from a bare model call and a growing prompt, here is a plain structure to move to. It is not fancy and that is the point.
Split the work into stages you can look at one at a time. Gather is where you pull candidate material from your sources. Select is where you rank, deduplicate, and drop. Assemble is where you place the surviving pieces into the window in a fixed order. Answer is the model call itself. Keeping these separate means that when an answer is bad you can tell which stage let you down.
Give each stage a hard limit and log what it emitted. When something goes wrong in production, the logs from Select and Assemble tell you almost everything: what was retrieved, what survived, and exactly what text the model saw. Without that record you are guessing, and guessing about a non-deterministic system is a slow way to spend a week.
Start there. You can add re-ranking models and clever compression later, once the plain version is running and you can measure what it does.