How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo

For a long time, the pain point in reinforcement learning research has been that meaningful metrics often emerge only after the entire experimental infrastructure is set up. Configuring environments, managing dependencies, handling errors, monitoring runs, and analyzing results — these repetitive tasks consume a significant amount of researcher time. Since 2025, with the advent of frontier models like GPT 5.5, coding agents have begun to demonstrate the ability to tackle such complex workflows.

On July 14, 2026, the NVIDIA developer blog published a detailed tutorial showing how to use a Codex agent (based on GPT 5.5) combined with NeMo RL and NeMo Gym to build a 'autoresearch' workflow that requires no user coding. The workflow is driven by the open-source Autoresearch project initiated by Andrej Karpathy, aiming to let agents handle repetitive setup and iteration tasks while human researchers focus on setting goals, reviewing milestones, and making final decisions.

The workflow is built on two open-source libraries from the NVIDIA NeMo framework: NeMo RL and NeMo Gym. NeMo RL, based on AutoModel, Megatron-Bridge, and vLLM, supports post-training processes such as GRPO, DPO, and SFT, is configuration-driven, and scales from small validation to distributed training. NeMo Gym provides interactive environments where models learn through real-time generated experiences and rewards.

The agent itself (Codex + GPT 5.5) has strong reasoning, code navigation, and tool-use capabilities. To adapt to local operational norms (e.g., checkpoint paths, metric authority, session recovery), the team equipped it with three complementary 'skills': Brev-etiquette handles system operation norms (keeping the repository clean, correctly placing large files, securely handling keys); session-memory maintains persistent session logs (recording goals, subtasks, decisions, progress); and autoresearch implements the experiment loop (preserving goals, establishing baselines, creating branches for each hypothesis, recording attempts, monitoring stop rules, summarizing results). These skills, encoded as structured reusable instructions, encapsulate operational context and institutional knowledge, making the agent's research loop more reproducible.

The tutorial demonstrates the agent's capabilities through three steps. First, the agent launches and validates the full-stack environment on an NVIDIA Brev GPU instance (equipped with a single NVIDIA L40S 48 GB GPU): installing dependencies, configuring VS Code and the Codex plugin, and validating the end-to-end training workflow of NeMo RL and NeMo Gym.

The second step is goal-driven autonomous research: the agent autonomously creates a novel NeMo Gym visual counting environment, then trains the Qwen3-VL-2B-Instruct model on this task, improving accuracy from 25.0% to 96.9%. This process is entirely self-directed by the agent: it analyzes baselines, proposes hypotheses, edits code, launches experiments, analyzes metrics, and retains effective improvements.

The third step showcases paper-to-code ability: the agent reads a research paper, forms an implementation plan, translates the offline policy RL algorithm into code, adds tests, and launches a 10-hour validation training activity. Throughout the process, the agent demonstrates understanding of the paper's content and the ability to translate it into runnable code. It should be noted that final decisions and strategic guidance remain under human control.

Despite the exciting results, it is important to recognize the limitations. First, the current workflow relies on a frontier model (GPT 5.5) and specific hardware (NVIDIA L40S 48 GB GPU), and the skillset (Brev-etiquette, session-memory) is tailored to a specific platform (Brev instances); generalizability remains to be verified.

Second, the long-term reliability of the agent has not been fully tested: issues such as context compression during extended runs, disconnection recovery, and goal drift are partially mitigated by session-memory skills, but stable operations over large-scale, multi-experiment scenarios require more evidence.

Additionally, while paper-to-code translation has been achieved, its quality still requires human review; the agent's grasp of mathematical details and subtle assumptions in papers is unclear. Finally, this work primarily focuses on the reinforcement learning domain (especially RL for VLM), and its transferability to other ML branches (e.g., supervised learning, generative models) remains to be explored.