Local Autonomous Coding Analyst & Self-Correcting Task Pipeline

Architected and engineered a local, multi-threaded code optimization and data analysis pipeline designed to execute untrusted LLM-generated code safely within strict resource constraints. Built an integrated state-management engine that coordinates autonomous code generation, sandbox execution, and dynamic self-correction loops entirely on an 8GB RAM Windows architecture. The platform applies localized execution telemetry to catch runtime errors, feeding automated traceback logs back into a quantized DeepSeek model for recursive code refinement, securing reliable script execution without crashing local hosting hardware.

Metrics at a Glance:

Pipeline Orchestration: 100% Sequential Multi-Thread Isolation (RAM Lock Protection)
Compute Optimization: Model footprint restricted to < 2.0GB (8GB System Guard)
Database Operations: Non-blocking asynchronous reading via SQLite WAL-Mode
I/O Latency: 10x process speedup by eliminating active virtual environment recreation

Key Architectural Achievements:

  • Self-Correcting Autonomous Orchestration: Engineered an iterative state-management loop that extracts, validates, and runs model-generated Python scripts, automatically capturing runtime failures and formatting targeted error injections for recursive code self-healing.
  • Process-Isolated Execution Sandbox: Designed a secure, lightweight runtime sandbox that uses localized regex import extraction and defensive file-handling to auto-resolve packages and execute code within strict timeout boundaries.
  • Single-Process Async Architecture: Implemented a single-process background execution queue using RQ's SimpleWorker framework to bypass POSIX process-forking constraints on Windows 10, preventing system thread exhaustion.
  • Low-Footprint Frontend Engineering: Developed a dual-column Streamlit management dashboard leveraging structural UI fragments to decouple user data entry from active 3-second queue polling loops, eliminating input lag.
  • Memory Protection & Recovery: Built custom garbage collection lifecycle hooks (gc.collect()) and strict context window truncation filters to aggressively flush residual token memory and prevent long-lived RAM creep.
Python 3.13 LangChain-Ollama Redis Queue (RQ) SQLite (WAL) Streamlit Process Subprocess Sandbox Asynchronous Data Engineering Architectural Resource Optimization

Self-Healing Autonomous Generative AI Web Extraction Engine

Architected and engineered an intelligent, autonomous web scraping agent built to eliminate data pipeline fragility caused by structural website updates and script execution crashes. Integrated a local DeepSeek-Coder large language model via LangChain to dynamically generate high-quality Python automation code at runtime based on natural language task profiles. Implemented a robust, self-correcting feedback loop that captures sandbox compilation runtime exceptions and missing dependencies, autonomously executing real-time recovery strategies such as automated package installations and algorithmic code rewrites to deliver deterministic data payloads.

Metrics at a Glance:

Model Architecture: DeepSeek-Coder:6.7b
Execution Environment: Isolated Sandbox Wrapper
Correction Loop Timeout: 45s Max / Attempt
Pipeline Orchestration: LangChain Expression Language (LCEL)

Key Architectural Achievements:

  • Dynamic Self-Correction Engine: Developed a comprehensive exception-handling layer that monitors subprocess execution state, intercepting standard errors to fuel a prompt-driven self-healing loop that rewrites failing scripts on the fly.
  • Universal Extraction Framework Contract: Authored a multi-strategy system prompt matrix enforcing deterministic code compliance, instructing the generative model to reactively choose between BeautifulSoup for static grids and Playwright for dynamic web components.
  • Automated Dependency Recovery: Engineered a regex-based pattern matching module capable of diagnosing missing external libraries from raw stack traces, routing tasks to a subprocess routine that triggers automated package setups without breaking runtime.
  • Asynchronous Telemetry Streaming: Formulated an optimized Python yield-generator network to stream live console output lines directly from the background compilation space straight into the client view, completely preventing dashboard freezing.
  • Intelligent Data Post-Processing: Programmed an analytical cleansing pipeline that extracts JSON or CSV markers from raw shell logs, automatically purging formatting clutter and currency symbols to build casted numeric structures for visualization.
Python Generative AI (GenAI) Large Language Models (LLMs) LangChain Ollama Streamlit Playwright Subprocess Sandbox Architecture Self-Healing Code Plotly Analytics
← Back to Main Portfolio