Implement Kitty graphics protocol for background image rendering
- Added `kitty_gfx` module to handle loading, scaling, and displaying background images using the Kitty graphics protocol. - Integrated background image handling into the main application loop, allowing dynamic updates based on configuration. - Enhanced terminal rendering to support transparent backgrounds when a Kitty image is active. - Updated IPC server to manage client connections and messages, including handling background image settings. - Modified `PtyTerminal` to accept additional environment variables during shell spawning. - Improved rendering logic to support popup dialogs and terminal background color customization.
This commit is contained in:
376
SOCKET_API.md
Normal file
376
SOCKET_API.md
Normal file
@@ -0,0 +1,376 @@
|
||||
# TUI-WM Socket API
|
||||
|
||||
TUI-WM exposes a Unix domain socket that allows other applications to interact with the window manager: spawn windows, show popup dialogs, list or close windows, and receive rendered frames.
|
||||
|
||||
## Overview
|
||||
|
||||
The protocol uses a simple framing format: every message is preceded by a 4-byte big-endian `u32` indicating the byte length of the JSON payload, followed by the JSON itself.
|
||||
|
||||
```
|
||||
[ 4 bytes: length (big-endian u32) ][ N bytes: JSON ]
|
||||
```
|
||||
|
||||
All messages are JSON objects with a mandatory `"type"` field (snake_case) that discriminates the message kind.
|
||||
|
||||
## Connecting
|
||||
|
||||
### Environment variables
|
||||
|
||||
When TUI-WM spawns a terminal inside a window it sets two environment variables for the child process:
|
||||
|
||||
| Variable | Example value | Description |
|
||||
|-------------------|----------------------------|------------------------------------------|
|
||||
| `TUI_WM_SOCKET` | `/run/user/1000/tui-wm.sock` | Path to the Unix socket |
|
||||
| `TUI_WM_WINDOW_ID`| `3` | The window ID of the containing window |
|
||||
|
||||
### Finding the socket path
|
||||
|
||||
If the environment variables are not set (e.g. you are connecting externally), the socket is located at:
|
||||
|
||||
- `$XDG_RUNTIME_DIR/tui-wm.sock` (preferred)
|
||||
- `/tmp/tui-wm-$USER.sock` (fallback)
|
||||
- `/tmp/tui-wm.sock` (last resort)
|
||||
|
||||
### Client roles
|
||||
|
||||
Every client must identify itself as one of two roles in the initial `hello` message:
|
||||
|
||||
- **`display`** — receives rendered frames (ANSI) from the server and forwards input back.
|
||||
- **`app`** — sends commands to the window manager (spawn windows, show popups, etc.) and receives responses.
|
||||
|
||||
A client can act as both simultaneously by choosing either role; `display` clients will receive `frame` messages in addition to any command responses.
|
||||
|
||||
## Handshake
|
||||
|
||||
Every connection must begin with a `hello` message. The server replies with `hello_ok`.
|
||||
|
||||
### Client → Server: `hello`
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "hello",
|
||||
"role": "display",
|
||||
"width": 220,
|
||||
"height": 50,
|
||||
"window_id": null
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------------|-----------------|----------|---------------------------------------------------|
|
||||
| `role` | `"display"` \| `"app"` | Yes | Client role |
|
||||
| `width` | integer (u16) | Yes | Client terminal width (used for frame sizing) |
|
||||
| `height` | integer (u16) | Yes | Client terminal height |
|
||||
| `window_id` | integer \| null | No | The window ID from `TUI_WM_WINDOW_ID` if available |
|
||||
|
||||
### Server → Client: `hello_ok`
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "hello_ok",
|
||||
"version": "0.1.0",
|
||||
"socket_path": "/run/user/1000/tui-wm.sock"
|
||||
}
|
||||
```
|
||||
|
||||
## Client → Server messages
|
||||
|
||||
### `input`
|
||||
|
||||
Forward raw terminal input bytes to the focused window. Bytes are hex-encoded.
|
||||
|
||||
```json
|
||||
{ "type": "input", "data": "1b5b41" }
|
||||
```
|
||||
|
||||
The example sends the bytes `0x1b 0x5b 0x41` (Up arrow).
|
||||
|
||||
### `resize`
|
||||
|
||||
Notify the server that the display client's terminal has been resized.
|
||||
|
||||
```json
|
||||
{ "type": "resize", "width": 200, "height": 48 }
|
||||
```
|
||||
|
||||
### `spawn_window`
|
||||
|
||||
Ask the server to open a new terminal window running the given command.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "spawn_window",
|
||||
"command": "/bin/bash",
|
||||
"request_id": "req-1"
|
||||
}
|
||||
```
|
||||
|
||||
The server responds with `window_opened`.
|
||||
|
||||
### `spawn_popup`
|
||||
|
||||
Show a modal popup dialog with a message and configurable buttons. The server responds with `popup_result` once the user dismisses the dialog.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "spawn_popup",
|
||||
"message": "Vill du avsluta?",
|
||||
"buttons": ["Ja", "Nej"],
|
||||
"request_id": "req-2"
|
||||
}
|
||||
```
|
||||
|
||||
If `buttons` is omitted it defaults to `["Ja", "Nej"]`.
|
||||
|
||||
### `list_windows`
|
||||
|
||||
Request a list of all open windows.
|
||||
|
||||
```json
|
||||
{ "type": "list_windows", "request_id": "req-3" }
|
||||
```
|
||||
|
||||
### `close_window`
|
||||
|
||||
Close the window with the given ID.
|
||||
|
||||
```json
|
||||
{ "type": "close_window", "window_id": 3, "request_id": "req-4" }
|
||||
```
|
||||
|
||||
### `set_background`
|
||||
|
||||
Set or remove the desktop background image (requires Kitty graphics protocol support in the host terminal).
|
||||
|
||||
```json
|
||||
{ "type": "set_background", "path": "/path/to/image.jpg", "save": true, "request_id": "req-5" }
|
||||
```
|
||||
|
||||
- `path` — Path to image file (JPEG/PNG/GIF/BMP/WebP), or `null` to remove the background.
|
||||
- `save` — If `true`, the change is persisted to `config.toml`.
|
||||
|
||||
## Server → Client messages
|
||||
|
||||
### `hello_ok`
|
||||
|
||||
See handshake section above.
|
||||
|
||||
### `frame`
|
||||
|
||||
Sent to `display` clients every render cycle (~60 fps). The `data` field is a hex-encoded ANSI escape sequence string that, when written to a terminal, renders the full current state of the TUI-WM screen.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "frame",
|
||||
"width": 200,
|
||||
"height": 48,
|
||||
"min_width": 200,
|
||||
"min_height": 48,
|
||||
"data": "1b5b306d1b5b324a..."
|
||||
}
|
||||
```
|
||||
|
||||
To display a frame, decode `data` from hex and write the raw bytes to stdout.
|
||||
|
||||
### `window_opened`
|
||||
|
||||
Response to `spawn_window`.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "window_opened",
|
||||
"id": 5,
|
||||
"request_id": "req-1"
|
||||
}
|
||||
```
|
||||
|
||||
### `popup_result`
|
||||
|
||||
Response to `spawn_popup` — sent when the user selects a button or presses Escape.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "popup_result",
|
||||
"button": "Ja",
|
||||
"button_index": 0,
|
||||
"request_id": "req-2"
|
||||
}
|
||||
```
|
||||
|
||||
### `window_list`
|
||||
|
||||
Response to `list_windows`.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "window_list",
|
||||
"windows": [
|
||||
{ "id": 1, "x": 2, "y": 2, "width": 82, "height": 26, "title": "bash" },
|
||||
{ "id": 3, "x": 6, "y": 4, "width": 82, "height": 26, "title": "nvim" }
|
||||
],
|
||||
"request_id": "req-3"
|
||||
}
|
||||
```
|
||||
|
||||
### `error`
|
||||
|
||||
Sent when a request cannot be fulfilled.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "error",
|
||||
"message": "Window not found",
|
||||
"request_id": "req-4"
|
||||
}
|
||||
```
|
||||
|
||||
## Code examples
|
||||
|
||||
### Bash — show a popup and read the result
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
SOCKET="${TUI_WM_SOCKET:-/tmp/tui-wm.sock}"
|
||||
|
||||
# Build the hello + spawn_popup messages
|
||||
send_msg() {
|
||||
local json="$1"
|
||||
local len=${#json}
|
||||
# Write 4-byte big-endian length followed by JSON
|
||||
printf "$(printf '\\x%02x\\x%02x\\x%02x\\x%02x' \
|
||||
$((len >> 24 & 0xff)) $((len >> 16 & 0xff)) \
|
||||
$((len >> 8 & 0xff)) $((len & 0xff)))"
|
||||
printf '%s' "$json"
|
||||
}
|
||||
|
||||
{
|
||||
send_msg '{"type":"hello","role":"app","width":80,"height":24}'
|
||||
send_msg '{"type":"spawn_popup","message":"Continue?","buttons":["Yes","No"],"request_id":"r1"}'
|
||||
} | nc -U "$SOCKET" | python3 -c "
|
||||
import sys, struct, json
|
||||
while True:
|
||||
hdr = sys.stdin.buffer.read(4)
|
||||
if len(hdr) < 4:
|
||||
break
|
||||
n = struct.unpack('>I', hdr)[0]
|
||||
msg = json.loads(sys.stdin.buffer.read(n))
|
||||
if msg.get('type') == 'popup_result':
|
||||
print('User chose:', msg['button'])
|
||||
break
|
||||
"
|
||||
```
|
||||
|
||||
### Python — connect as display client and render frames
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import os, socket, struct, json, sys
|
||||
|
||||
SOCKET_PATH = os.environ.get("TUI_WM_SOCKET", "/tmp/tui-wm.sock")
|
||||
|
||||
def send(sock, msg: dict):
|
||||
data = json.dumps(msg).encode()
|
||||
sock.sendall(struct.pack(">I", len(data)) + data)
|
||||
|
||||
def recv(sock) -> dict:
|
||||
hdr = b""
|
||||
while len(hdr) < 4:
|
||||
hdr += sock.recv(4 - len(hdr))
|
||||
n = struct.unpack(">I", hdr)[0]
|
||||
buf = b""
|
||||
while len(buf) < n:
|
||||
buf += sock.recv(n - len(buf))
|
||||
return json.loads(buf)
|
||||
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
|
||||
s.connect(SOCKET_PATH)
|
||||
|
||||
# Handshake
|
||||
term_size = os.get_terminal_size()
|
||||
send(s, {"type": "hello", "role": "display",
|
||||
"width": term_size.columns, "height": term_size.lines})
|
||||
hello_ok = recv(s)
|
||||
assert hello_ok["type"] == "hello_ok"
|
||||
|
||||
# Render incoming frames
|
||||
sys.stdout.write("\033[?1049h\033[?25l") # alternate screen, hide cursor
|
||||
sys.stdout.flush()
|
||||
try:
|
||||
while True:
|
||||
msg = recv(s)
|
||||
if msg["type"] == "frame":
|
||||
ansi = bytes.fromhex(msg["data"])
|
||||
sys.stdout.buffer.write(ansi)
|
||||
sys.stdout.buffer.flush()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
sys.stdout.write("\033[?1049l\033[?25h") # restore
|
||||
sys.stdout.flush()
|
||||
```
|
||||
|
||||
### Python — spawn a window and list windows
|
||||
|
||||
```python
|
||||
import os, socket, struct, json
|
||||
|
||||
SOCKET_PATH = os.environ.get("TUI_WM_SOCKET", "/tmp/tui-wm.sock")
|
||||
|
||||
def send(sock, msg):
|
||||
data = json.dumps(msg).encode()
|
||||
sock.sendall(struct.pack(">I", len(data)) + data)
|
||||
|
||||
def recv(sock):
|
||||
hdr = b""
|
||||
while len(hdr) < 4:
|
||||
hdr += sock.recv(4 - len(hdr))
|
||||
n = struct.unpack(">I", hdr)[0]
|
||||
buf = b""
|
||||
while len(buf) < n:
|
||||
buf += sock.recv(n - len(buf))
|
||||
return json.loads(buf)
|
||||
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
|
||||
s.connect(SOCKET_PATH)
|
||||
send(s, {"type": "hello", "role": "app", "width": 80, "height": 24})
|
||||
recv(s) # hello_ok
|
||||
|
||||
# Spawn a window
|
||||
send(s, {"type": "spawn_window", "command": "/bin/bash", "request_id": "a"})
|
||||
opened = recv(s)
|
||||
print("Opened window id:", opened["id"])
|
||||
|
||||
# List all windows
|
||||
send(s, {"type": "list_windows", "request_id": "b"})
|
||||
wlist = recv(s)
|
||||
for w in wlist["windows"]:
|
||||
print(f" Window {w['id']}: {w['title']} at ({w['x']},{w['y']}) {w['width']}x{w['height']}")
|
||||
```
|
||||
|
||||
## Operating modes
|
||||
|
||||
TUI-WM can be started in three modes:
|
||||
|
||||
### Standalone (default)
|
||||
|
||||
```
|
||||
tui-wm
|
||||
```
|
||||
|
||||
Runs the full TUI window manager in the current terminal and simultaneously starts a Unix socket server. Other applications can connect to the socket while TUI-WM is running.
|
||||
|
||||
### Daemon (`-d`)
|
||||
|
||||
```
|
||||
tui-wm -d
|
||||
```
|
||||
|
||||
Runs headless — no terminal rendering. State is managed entirely via the socket. Frames are rendered to an internal buffer and sent to any connected `display` clients. Ideal for running TUI-WM as a background service.
|
||||
|
||||
### Client (`-c`)
|
||||
|
||||
```
|
||||
tui-wm -c [socket-path]
|
||||
```
|
||||
|
||||
Connects to an existing TUI-WM server (standalone or daemon) as a display client. The client renders received frames and forwards keyboard/resize events to the server. If `socket-path` is omitted, the default path is used. Disconnect with **Ctrl+Shift+Q**.
|
||||
Reference in New Issue
Block a user