Files
TUI-WM/SOCKET_API.md
Bjorn Blomberg f0bed7a14e Full remote input parity: forward mouse and structured key events over IPC
- New key_event/mouse_event client messages (crossterm serde) that run
  through the complete WM event handling on the server — remote clients
  get window focus, drag/resize and mouse forwarding into virtual
  terminals (SGR/UTF-8/X10) identical to standalone mode
- tui-wm -c now forwards mouse events; previously only keyboard worked
  remotely
- IPC unit tests + end-to-end integration test (tests/integration.py):
  daemon, frames, click/scroll SGR forwarding, popups, TUI-FM in a
  window — 12/12 passing
- Document the new messages in SOCKET_API.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 19:50:27 +02:00

11 KiB

TUI-WM Socket API

TUI-WM exposes a Unix domain socket that allows other applications to interact with the window manager: spawn windows, show popup dialogs, list or close windows, and receive rendered frames.

Overview

The protocol uses a simple framing format: every message is preceded by a 4-byte big-endian u32 indicating the byte length of the JSON payload, followed by the JSON itself.

[ 4 bytes: length (big-endian u32) ][ N bytes: JSON ]

All messages are JSON objects with a mandatory "type" field (snake_case) that discriminates the message kind.

Connecting

Environment variables

When TUI-WM spawns a terminal inside a window it sets two environment variables for the child process:

Variable Example value Description
TUI_WM_SOCKET /run/user/1000/tui-wm.sock Path to the Unix socket
TUI_WM_WINDOW_ID 3 The window ID of the containing window

Finding the socket path

If the environment variables are not set (e.g. you are connecting externally), the socket is located at:

  • $XDG_RUNTIME_DIR/tui-wm.sock (preferred)
  • /tmp/tui-wm-$USER.sock (fallback)
  • /tmp/tui-wm.sock (last resort)

Client roles

Every client must identify itself as one of two roles in the initial hello message:

  • display — receives rendered frames (ANSI) from the server and forwards input back.
  • app — sends commands to the window manager (spawn windows, show popups, etc.) and receives responses.

A client can act as both simultaneously by choosing either role; display clients will receive frame messages in addition to any command responses.

Handshake

Every connection must begin with a hello message. The server replies with hello_ok.

Client → Server: hello

{
  "type": "hello",
  "role": "display",
  "width": 220,
  "height": 50,
  "window_id": null
}
Field Type Required Description
role "display" | "app" Yes Client role
width integer (u16) Yes Client terminal width (used for frame sizing)
height integer (u16) Yes Client terminal height
window_id integer | null No The window ID from TUI_WM_WINDOW_ID if available

Server → Client: hello_ok

{
  "type": "hello_ok",
  "version": "0.1.0",
  "socket_path": "/run/user/1000/tui-wm.sock"
}

Client → Server messages

input

Forward raw terminal input bytes to the focused window. Bytes are hex-encoded.

{ "type": "input", "data": "1b5b41" }

The example sends the bytes 0x1b 0x5b 0x41 (Up arrow).

key_event

Forward a structured keyboard event. Unlike input, the event runs through the full window-manager event handling: global keybinds, dialogs, focus and PTY forwarding — exactly as if the key was pressed locally in standalone mode. This is what tui-wm -c sends for every keypress.

{
  "type": "key_event",
  "event": { "code": { "Char": "a" }, "modifiers": "CONTROL", "kind": "Press", "state": "" }
}

code is either { "Char": "<tecken>" } or a string for special keys ("Enter", "Esc", "Tab", "Backspace", "Up", "Down", "Left", "Right", "Home", "End", "PageUp", "PageDown", "Delete", ...). modifiers is a |-separated string of SHIFT, CONTROL, ALT (empty string = none).

mouse_event

Forward a structured mouse event. Gives remote clients full mouse parity with standalone mode: window focus, title-bar drag, resize, and forwarding into the virtual terminals with correct SGR/UTF-8/X10 encoding and coordinate translation. This is what tui-wm -c sends for every mouse event.

{
  "type": "mouse_event",
  "event": { "kind": { "Down": "Left" }, "column": 10, "row": 5, "modifiers": "" }
}

kind is one of { "Down": <btn> }, { "Up": <btn> }, { "Drag": <btn> }, "Moved", "ScrollUp", "ScrollDown" where <btn> is "Left", "Right" or "Middle". Coordinates are 0-based frame coordinates.

resize

Notify the server that the display client's terminal has been resized.

{ "type": "resize", "width": 200, "height": 48 }

spawn_window

Ask the server to open a new terminal window running the given command.

{
  "type": "spawn_window",
  "command": "/bin/bash",
  "request_id": "req-1"
}

The server responds with window_opened.

spawn_popup

Show a modal popup dialog with a message and configurable buttons. The server responds with popup_result once the user dismisses the dialog.

{
  "type": "spawn_popup",
  "message": "Vill du avsluta?",
  "buttons": ["Ja", "Nej"],
  "request_id": "req-2"
}

If buttons is omitted it defaults to ["Ja", "Nej"].

list_windows

Request a list of all open windows.

{ "type": "list_windows", "request_id": "req-3" }

close_window

Close the window with the given ID.

{ "type": "close_window", "window_id": 3, "request_id": "req-4" }

set_background

Set or remove the desktop background image (requires Kitty graphics protocol support in the host terminal).

{ "type": "set_background", "path": "/path/to/image.jpg", "save": true, "request_id": "req-5" }
  • path — Path to image file (JPEG/PNG/GIF/BMP/WebP), or null to remove the background.
  • save — If true, the change is persisted to config.toml.

Server → Client messages

hello_ok

See handshake section above.

frame

Sent to display clients every render cycle (~60 fps). The data field is a hex-encoded ANSI escape sequence string that, when written to a terminal, renders the full current state of the TUI-WM screen.

{
  "type": "frame",
  "width": 200,
  "height": 48,
  "min_width": 200,
  "min_height": 48,
  "data": "1b5b306d1b5b324a..."
}

To display a frame, decode data from hex and write the raw bytes to stdout.

window_opened

Response to spawn_window.

{
  "type": "window_opened",
  "id": 5,
  "request_id": "req-1"
}

popup_result

Response to spawn_popup — sent when the user selects a button or presses Escape.

{
  "type": "popup_result",
  "button": "Ja",
  "button_index": 0,
  "request_id": "req-2"
}

window_list

Response to list_windows.

{
  "type": "window_list",
  "windows": [
    { "id": 1, "x": 2, "y": 2, "width": 82, "height": 26, "title": "bash" },
    { "id": 3, "x": 6, "y": 4, "width": 82, "height": 26, "title": "nvim" }
  ],
  "request_id": "req-3"
}

error

Sent when a request cannot be fulfilled.

{
  "type": "error",
  "message": "Window not found",
  "request_id": "req-4"
}

Code examples

Bash — show a popup and read the result

#!/usr/bin/env bash
SOCKET="${TUI_WM_SOCKET:-/tmp/tui-wm.sock}"

# Build the hello + spawn_popup messages
send_msg() {
    local json="$1"
    local len=${#json}
    # Write 4-byte big-endian length followed by JSON
    printf "$(printf '\\x%02x\\x%02x\\x%02x\\x%02x' \
        $((len >> 24 & 0xff)) $((len >> 16 & 0xff)) \
        $((len >>  8 & 0xff)) $((len       & 0xff)))"
    printf '%s' "$json"
}

{
    send_msg '{"type":"hello","role":"app","width":80,"height":24}'
    send_msg '{"type":"spawn_popup","message":"Continue?","buttons":["Yes","No"],"request_id":"r1"}'
} | nc -U "$SOCKET" | python3 -c "
import sys, struct, json
while True:
    hdr = sys.stdin.buffer.read(4)
    if len(hdr) < 4:
        break
    n = struct.unpack('>I', hdr)[0]
    msg = json.loads(sys.stdin.buffer.read(n))
    if msg.get('type') == 'popup_result':
        print('User chose:', msg['button'])
        break
"

Python — connect as display client and render frames

#!/usr/bin/env python3
import os, socket, struct, json, sys

SOCKET_PATH = os.environ.get("TUI_WM_SOCKET", "/tmp/tui-wm.sock")

def send(sock, msg: dict):
    data = json.dumps(msg).encode()
    sock.sendall(struct.pack(">I", len(data)) + data)

def recv(sock) -> dict:
    hdr = b""
    while len(hdr) < 4:
        hdr += sock.recv(4 - len(hdr))
    n = struct.unpack(">I", hdr)[0]
    buf = b""
    while len(buf) < n:
        buf += sock.recv(n - len(buf))
    return json.loads(buf)

with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
    s.connect(SOCKET_PATH)

    # Handshake
    term_size = os.get_terminal_size()
    send(s, {"type": "hello", "role": "display",
             "width": term_size.columns, "height": term_size.lines})
    hello_ok = recv(s)
    assert hello_ok["type"] == "hello_ok"

    # Render incoming frames
    sys.stdout.write("\033[?1049h\033[?25l")  # alternate screen, hide cursor
    sys.stdout.flush()
    try:
        while True:
            msg = recv(s)
            if msg["type"] == "frame":
                ansi = bytes.fromhex(msg["data"])
                sys.stdout.buffer.write(ansi)
                sys.stdout.buffer.flush()
    except KeyboardInterrupt:
        pass
    finally:
        sys.stdout.write("\033[?1049l\033[?25h")  # restore
        sys.stdout.flush()

Python — spawn a window and list windows

import os, socket, struct, json

SOCKET_PATH = os.environ.get("TUI_WM_SOCKET", "/tmp/tui-wm.sock")

def send(sock, msg):
    data = json.dumps(msg).encode()
    sock.sendall(struct.pack(">I", len(data)) + data)

def recv(sock):
    hdr = b""
    while len(hdr) < 4:
        hdr += sock.recv(4 - len(hdr))
    n = struct.unpack(">I", hdr)[0]
    buf = b""
    while len(buf) < n:
        buf += sock.recv(n - len(buf))
    return json.loads(buf)

with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
    s.connect(SOCKET_PATH)
    send(s, {"type": "hello", "role": "app", "width": 80, "height": 24})
    recv(s)  # hello_ok

    # Spawn a window
    send(s, {"type": "spawn_window", "command": "/bin/bash", "request_id": "a"})
    opened = recv(s)
    print("Opened window id:", opened["id"])

    # List all windows
    send(s, {"type": "list_windows", "request_id": "b"})
    wlist = recv(s)
    for w in wlist["windows"]:
        print(f"  Window {w['id']}: {w['title']} at ({w['x']},{w['y']}) {w['width']}x{w['height']}")

Operating modes

TUI-WM can be started in three modes:

Standalone (default)

tui-wm

Runs the full TUI window manager in the current terminal and simultaneously starts a Unix socket server. Other applications can connect to the socket while TUI-WM is running.

Daemon (-d)

tui-wm -d

Runs headless — no terminal rendering. State is managed entirely via the socket. Frames are rendered to an internal buffer and sent to any connected display clients. Ideal for running TUI-WM as a background service.

Client (-c)

tui-wm -c [socket-path]

Connects to an existing TUI-WM server (standalone or daemon) as a display client. The client renders received frames and forwards keyboard/resize events to the server. If socket-path is omitted, the default path is used. Disconnect with Ctrl+Shift+Q.