Engineering Trade-offs & Decisions
Building an AI-driven code intelligence platform at scale requires strict trade-offs between latency, cost, and analytical depth. We opted for a "Progressive Delivery" model powered by a multi-model routing strategy on Amazon Bedrock.
AI Grounding Logic
LLMs have a tendency to hallucinate architectural complexity when reading code. To counter this, we implemented strict Grounding Assertions. The system is programmed to distinguish between user-written code and boilerplate framework code (e.g., standard React setup, Express middleware defaults).
Every architectural claim made by the Synthesis Agent MUST reference a specific file path and line number. If the assertion engine detects a claim without verifiable origin in the Context Map, the claim is stripped. This ensures recruiters get an honest, factual representation of the candidate's actual work.
Multi-Model AI Routing
Cost optimization is critical when processing 50K+ tokens per repository. We use a dynamic routing strategy via Amazon Bedrock:
- Stage 1 (Code Review) & Stage 2 (Intelligence)We route to Claude 3.5 Sonnet / Mistral Large 3 for high-reasoning tasks. These models excel at synthesizing architectural trade-offs from raw code but are expensive. We offset costs by parallelizing narrow queries rather than asking one massive question.
- Stage 3 (Interview Real-Time Evaluation)We step down to faster inference models for the interactive websocket loop. The context window is small (just the current question and answer), requiring low latency rather than deep code synthesis.
Trade-off: Serverless Cold Starts vs Idle Costs
Running this on provisioned containers (ECS/EKS) would solve cold starts but incur massive idle costs given the bursty nature of resume processing. We chose AWS Lambda for scale-to-zero capabilities. The trade-off is a potential 1-3 second cold start penalty on the initial repository clone. We mask this latency from the user using an optimistic UI loading sequence on the frontend.
function validateClaim(claim) {
if (!claim.filePath || !claim.lineRefs) {
return { valid: false, reason: 'unverifiable' }
}
const mapNode = contextMap.get(claim.filePath);
if (mapNode.isBoilerplate) {
return { valid: false, reason: 'framework_code' }
}
return { valid: true };
}