Final Proposal Data Gathering Engine

• Your agent searches a state broadband office website, identifies the most recent Final Proposal document, downloads it, and extracts structured data from PDF tables or CSV files.
• Your agent detects that a state has published a new revision since the last run. It downloads the update, compares key metrics (location counts, funding totals) to the prior version, and flags significant changes for human review.
• Your agent encounters a state with no public Final Proposal data. It searches for press releases, NTIA filings, or legislative updates to determine the state's expected timeline.
• A user asks "Which states have increased their location counts since their initial filing?" and the agent queries the normalized dataset to answer.

Your pipeline should produce normalized records with at least these fields per state:

1. Where is their Final Proposal data published? What format is it in?
2. How easy is it to extract the target schema fields from this format?
3. What inconsistencies do you notice across states (naming conventions, data granularity, file structures)?
4. Which fields from the target schema are commonly available, and which are rare or missing?
5. What would break if you tried to apply the same extraction logic to all 3 states?

Design and build an agentic workflow that gathers, extracts, normalizes, and validates Final Proposal data.

• Architecture diagram (Mermaid or equivalent) showing agents, tools, data flow (from raw to transformed data), and where humans can intervene.
• Clear agent boundaries. Use LLMs for reasoning (e.g., classifying document structure, extracting data from unstructured PDFs) and deterministic scripts for execution (e.g., downloading files, parsing CSVs, writing to the output schema).
• Clear tool definitions. The tools and services used by each agent, with their schemas and descriptions.
• State management. How context, intermediate results, and per-state progress are tracked.
• Failure handling. What happens when a source is unavailable, a PDF is unparseable, or extracted values look anomalous.

The collected data must be automatically audited. Design validation that catches real problems:

• Cross-field consistency checks (e.g., locations_to_fund + locations_excluded <= eligible_locations).
• Anomaly detection on funding amounts (e.g., a state reporting $0 or $999B).
• Completeness scoring per state (what percentage of target schema fields were populated?).
• Human-in-the-loop escalation for edge cases the agent can't resolve confidently.
• Completeness check to make sure your agent does not miss critical information or data.

Extend your pipeline beyond the seed states toward full U.S. coverage. Not every state will be easy: some publish clean CSVs, others bury data in PDFs, and some have nothing public at all. What we're looking for:

• Fallback strategies for hard cases. If your primary extraction method fails for a state, what's plan B? (e.g., LLM-assisted PDF parsing, manual schema mapping, scraping state portals)
• Graceful degradation. A state with only partial data should still produce a record with what's available, not fail silently.
• Categorization of failures. For states you can't process, explain why (no data published, format too complex, behind authentication, etc.) and what would be needed to resolve it.
• Coverage report. A summary showing: states fully processed, states partially processed (with which fields missing), and states not processable (with reason).

Using the data you've collected, document your core findings:

• How many states have approved Final Proposals vs. still in revision vs. no data?
• What is the total funding allocated across collected states? Average per location?
• Which providers appear most frequently? What share of locations go to LEO vs. non-LEO?
• What patterns or surprises did you find in the data?

Build a locally-hosted dashboard (Streamlit, Plotly Dash, etc.) that includes:

• Summary charts of funding allocation and state status.
• A map showing states color-coded by status or funding level.
• Location-level map overlay for states where coordinate data is available.
• Filtering by state, provider, or status.

Design (and optionally implement) a scheduling mechanism for weekly data refresh:

• How would you detect changes since the last run?
• How would you handle version tracking and changelog generation?
• What alerting would you set up for significant changes?

• Every AI tool you used (Claude, Copilot, Cursor, Codex, v0, etc.).
• What you used each tool for.
• 2–3 cases where you diverged from AI-generated output and why.

• Walk through your pipeline's handling of a specific state, from discovery to normalized output.
• Respond to a scenario change ("What if a state switches from PDF to an API?").
• Debug a suspicious data point from your own output.
• Explain how you'd handle 50 states with 5 different data formats each.

• GitHub repo (public, or invite-shared with @brookefzy, @am2222, and @tgibbons).
• README.md with a decision log (Decision → Alternatives → Reasoning → What you'd revisit).
• AI_TOOLS.md
• /docs — architecture diagram, Step 0 research findings, analysis rationale.
• /src — pipeline code, agent definitions, tool schemas.
• 5-minute Loom walkthrough (optional, encouraged).