codexctrl

codexctrl is a Windows command-line tool that lets Codex inspect and control desktop applications. It can read windows and UI elements, click, type, scroll, drag and take local screenshots. Compact responses and batches of known actions help reduce tool calls and context overhead without requiring the Computer Use plugin.

Quick install from Forgejo (Windows CMD)

1. Install the program

powershell.exe -NoProfile -Command "$ErrorActionPreference='Stop'; if (-not $env:LOCALAPPDATA) { throw 'LOCALAPPDATA missing' }; $ctrlBase=[IO.Path]::GetFullPath($env:LOCALAPPDATA).TrimEnd('\'); $ctrlSrc=[IO.Path]::GetFullPath((Join-Path $ctrlBase 'codexctrl-source')); if ([IO.Path]::GetDirectoryName($ctrlSrc) -ne $ctrlBase) { throw 'Unsafe source path' }; if (Test-Path -LiteralPath $ctrlSrc) { $ctrlItem=Get-Item -LiteralPath $ctrlSrc -Force; if (-not $ctrlItem.PSIsContainer -or ($ctrlItem.Attributes -band [IO.FileAttributes]::ReparsePoint)) { throw 'Source path must be a normal directory' }; if (Get-ChildItem -LiteralPath $ctrlSrc -Force -Recurse -Attributes ReparsePoint) { throw 'Source contains links; remove them manually' }; Remove-Item -LiteralPath $ctrlSrc -Recurse -Force }" && git clone --branch codex/standalone-cli --single-branch https://git.prov.one/sascha/codexctrl.git "%LOCALAPPDATA%\codexctrl-source" && call "%LOCALAPPDATA%\codexctrl-source\scripts\install-source.cmd"

Run in Command Prompt (CMD) on Windows 10/11 with Git, Windows PowerShell and the .NET 8 SDK installed. Use a terminal outside the source folder. The command deletes %LOCALAPPDATA%\codexctrl-source if it exists, clones a fresh checkout, then builds and installs under %LOCALAPPDATA%\Programs\codexctrl. Everything in that source folder, including uncommitted changes and local files, is removed on every run. The deletion checks the exact target path and rejects directory links. Other installation folders and Codex instruction files are not deleted. This route uses Forgejo source code and does not depend on GitHub release downloads.

Repository access: this Forgejo repository is currently private, so cloning requires authorized access. If the owner makes it public later, the same HTTPS command can be used without a Forgejo account or SSH key, provided anonymous access to public repositories is enabled on the server. The installation command does not change repository visibility or global Codex instructions.

2. Edit your global AGENTS.md yourself

Open %USERPROFILE%\.codex\AGENTS.md in a text editor and add the following block yourself, preserving your existing instructions. Replace YOUR_USERNAME with your Windows username and use the actual installed path if it differs.

## Windows desktop control
- Use `C:/Users/YOUR_USERNAME/AppData/Local/Programs/codexctrl/codexctrl.exe` for Windows desktop control.
- Read its `help` output when needed. Inspect unfamiliar targets before input.
- Prefer compact text output and batch known actions.
- Use Computer Use or @oai/sky only when I explicitly request it.
- If codexctrl fails, report the error instead of silently switching tools.

If CODEX_HOME is set, edit the file there instead. If AGENTS.override.md exists in that directory, review and edit that active file instead of the inactive AGENTS.md. Program installation never edits these files. Start a new Codex chat after saving your manual changes. See step 2 of the complete guide for opening the file and checking the executable path.

The independent C# executable uses Windows UI Automation, Win32 and SendInput. It does not require @oai/sky, MCP, Node.js or an OpenAI API key.

Optional prebuilt installer from GitHub

curl.exe -fL https://raw.githubusercontent.com/ol0f/codexctrl/main/install.cmd -o "%TEMP%\codexctrl-install.cmd" && call "%TEMP%\codexctrl-install.cmd"

Run this in Command Prompt (CMD) on 64-bit Windows 10/11. No administrator access or developer tools required. The installer downloads a standalone build with its .NET runtime, verifies its checksum, and installs it under %LOCALAPPDATA%\Programs\codexctrl. Then complete manual step 2 above. The installer never changes Codex instructions; the former --codex option has been removed.

Share the quick install guide with a colleague. After they manually configure AGENTS.md and start a new Codex chat, they can ask: “Use codexctrl to list my open windows.”

If GitHub downloads are unavailable, the project owner can share an offline ZIP directly. Extract it and run install.cmd in CMD, then complete manual step 2. The installer verifies and uses the adjacent executable without downloading it. Older v0.2.1 archives may still include the former optional setup feature; do not pass --codex to those installers.

What it does

• List windows and read bounded UI Automation observations.
• Click, focus, set field values, select items and toggle checkboxes.
• Send keyboard shortcuts and Unicode text without using the clipboard.
• Scroll, drag and run batches of known actions.
• Return compact JSON, optional observation differences and local screenshots.

Codex calls the executable through its normal command-line tool. No additional assistant tool connection is needed. Existing project instructions and explicit user requests can override a global preference.

Commands

Test an installed copy from CMD:

"%LOCALAPPDATA%\Programs\codexctrl\codexctrl.exe" windows
"%LOCALAPPDATA%\Programs\codexctrl\codexctrl.exe" help

The table below shows arguments to append to the executable. Replace HWND, REF and ID with actual returned values.

Command	Purpose
windows [--match TEXT]	List visible windows, process IDs and titles
inspect --window HWND	Read a compact accessibility observation
inspect --window HWND --match TEXT	Return matching elements only
inspect --window HWND --max 30 --chars 3000 --depth 4	Limit output size
inspect --window HWND --diff REF	Read changes since an earlier observation
inspect --window HWND --text	Include document text when TextPattern is available
click --ref REF --id ID	Invoke/select an element or click its current bounds
focus --ref REF --id ID	Focus an element
set --ref REF --id ID --text TEXT	Set an editable field value
toggle --ref REF --id ID	Toggle a supported checkbox
select --ref REF --id ID	Select a list item
click --window HWND --x 100 --y 200	Physical click; optional --button right --count 2
text --window HWND --text TEXT	Type Unicode into the focused control
key --window HWND --key ctrl+a	Send a keyboard shortcut
scroll --window HWND --x 100 --y 200 --dy -3	Scroll down three wheel steps; positive means up
drag --window HWND --x 20 --y 50 --to-x 200 --to-y 100	Drag with the left mouse button
screenshot --window HWND --out .local/image.png	Save a local PNG and return its path
batch --window HWND --file actions.json	Execute a known sequence on one window
ping	Read version, server process ID and pause status
pause / resume / stop	Pause input, resume input or stop the helper

Input commands support --observe to read the resulting state, and --wait 100 to wait before returning. The default input wait is 60 ms. Element references expire after 120 seconds and are invalidated by subsequent observations or inputs on that window.

Batch example

Save the following as actions.json, then run batch --window HWND --file actions.json --observe. Focus the intended text field first.

[
  {"op":"key","key":"ctrl+a"},
  {"op":"text","text":"Hello world"}
]

Batches allow 1–50 coordinate clicks, keyboard shortcuts, text inputs, scrolls or drags. They stop on the first error and report completed and failedIndex; they do not roll back or retry. Use individual inputs with observations when a transition is uncertain. Prefer a JSON file over inline JSON to avoid shell quoting issues.

Physical coordinates are relative to the outer window and measured in physical pixels. UIA b values use absolute screen coordinates. A resized screenshot returns scale; divide image coordinates by that scale before clicking.

Build from source

The development build requires the .NET 8 SDK. Unlike the installer release, its default output uses the installed .NET 8 Desktop Runtime.

scripts\build.cmd
codexctrl.cmd help
dotnet build native/CodexCtrl --warnaserror
dotnet run --project native/CodexCtrl.Tests

The local build is written to .local/bin/. To run the interactive integration checks, which open and close a dedicated test window:

dotnet run --project native/CodexCtrl.Tests -- --integration

The optional token-audit script and its tests require Node.js 22; the controller itself does not:

node --test test/usage-audit.test.mjs

Runtime behavior and limits

The first desktop command starts a hidden local helper. It accepts requests over a current-user named pipe, separately for each Windows session, and exits after 15 idle minutes. No TCP port, cloud service, telemetry or automatic screenshot upload is used.

Hold Escape to interrupt between batch operations, text chunks or drag steps. pause blocks later inputs when the server processes it. Neither can cancel a UI Automation provider that is already stuck; a CLI timeout does not guarantee that an in-flight operation has been cancelled. Inspect the state before retrying.

Use an unlocked local desktop. Elevated applications, UAC and protected desktops are not bypassed. Some games, canvases and Electron applications expose little accessibility information; use screenshots or known keyboard shortcuts when appropriate. Screenshots capture the visible window area and can include overlays. Password fields are excluded where the application's accessibility provider identifies them correctly. This is a powerful local input tool, not a security boundary.

Measurements and documentation

The initial implementation passed 24 checks, including real UI Automation input, Unicode typing, batching, screenshots and pause/resume. Local warm status calls took approximately 136 ms in an earlier test. Actual assistant latency and token costs depend on the task, context and required observations; they are not guaranteed by these native timings.

• Quick install guide — ready-to-share English instructions.
• Measured CLI/Computer Use token comparison.
• Architecture, test report and global setup — existing German technical notes.
• Changelog and agent instructions.

Version 0.1 used a Computer Use wrapper. It has been replaced; its source remains in Git history.