SSH Session Agent

Problem

After centralizing RDP sessions, the next natural question was the same one for Linux servers: who accessed a host through SSH, where did the connection come from, and how long did the session exist?

The implementation could not simply turn every sshd log line into a lifecycle event.

A normal SSH connection often produces multiple pieces of evidence for the same logical session:

Accepted publickey/password ... from <ip> port <port>
pam_unix(sshd:session): session opened for user <username>
pam_unix(sshd:session): session closed for user <username>

At the same time, very short sessions can open and close completely between two observations of current state.

The real problem is therefore identifying one logical session from sources that serve different purposes.

Solution

SSH Session Agent is the Linux collector for Remote Session API.

Version 0.1.0 combines:

• systemd-logind as the primary authority for persistent sessions;
• journald/OpenSSH as authentication and origin evidence;
• PAM open/close signals;
• the kernel boot_id to separate sessions across reboots;
• durable local spooling;
• periodic snapshots;
• the generic /api/v2 contract.

The Agent is written for Python 3.10+ and declares no external Python runtime dependencies.

Local architecture

The architecture avoids instrumenting the user’s shell or login process.

The Agent observes only metadata already produced by the operating system and OpenSSH.

Logind as the session authority

The collector queries:

loginctl list-sessions
loginctl show-session <id>

A session enters the SSH domain only when properties indicate a real remote SSH session, such as:

Remote=yes
Service=ssh or sshd
State=active or online

Its provider identity becomes:

provider_session_id = logind:<session-id>

That number is not treated as globally unique by itself. The API interprets it together with server identity, protocol, and boot_id.

Journald as evidence

The Agent reads the journal for ssh.service or sshd.service using JSON output and a persistent cursor.

From Accepted records, it keeps only the metadata needed for session correlation:

• authentication method;
• username;
• source IP;
• source TCP port;
• process PID;
• timestamp;
• journal cursor.

The remainder of the log line is deliberately discarded.

Public-key fingerprints and key material are therefore not persisted simply because they appeared in the original sshd message.

Why Accepted is not a LOGON by itself

If Accepted and pam session opened were emitted independently as lifecycle records, one connection could easily become two or more sessions in the central system.

The current model is different:

logind       -> proves the persistent session exists
Accepted     -> enriches it with IP, port, and method
PAM OPEN     -> improves the opening timestamp
PAM CLOSE    -> improves the closing timestamp

The result is at most one LOGON for a persistent SSH session.

Matching by PID

When a new logind session appears, the Agent tries to associate it with a recent authentication observation that has not already been claimed.

The strongest join is:

logind leader PID == sshd observation PID

When that join is unavailable, a more restricted fallback can use username and source IP within a short time window.

The goal is to avoid attaching an earlier authentication from the same user/address to the wrong current session.

Authentication matching window

The default configuration uses:

{
  "auth_match_window_seconds": 180
}

Old authentication observations are pruned from state so the cache does not grow indefinitely and stale evidence does not remain eligible for matching.

Ephemeral SSH sessions

Not every SSH connection remains open long enough to appear on the next logind poll.

A common example is:

ssh server 'short-command'

If journald contains the complete sequence:

Accepted
PAM OPEN
PAM CLOSE

and no persistent logind session claimed that observation, the Agent can reconstruct a deterministic pair:

LOGON
LOGOFF

The provider identity uses the sshd PID and a hash derived from the Accepted journal cursor.

This allows short-lived connections to be represented without depending entirely on poll timing.

Bootstrap without invented history

On first start, SSH sessions may already exist before the Agent begins observing the host.

Emitting a retroactive LOGON at that point would manufacture an event that was never actually observed.

Bootstrap therefore behaves like this:

first cycle
 -> observe current sessions
 -> send a snapshot
 -> persist state
 -> do not emit synthetic historical LOGONs

The same principle is applied to complete ephemeral evidence found during bootstrap.

The distinction between observed and inferred evidence is central to the design.

Kernel boot_id

Linux exposes the current boot identity at:

/proc/sys/kernel/random/boot_id

The Agent includes that value in events and snapshots.

When the boot changes:

• old logind IDs and PIDs are not treated as the same sessions;
• authentication evidence from the previous boot is not reused for matching;
• a snapshot of the new boot is forced immediately;
• the API can close previous sessions using end_reason=REBOOT.

The Agent does not fabricate stale LOGOFF events after a reboot.

SSH lifecycle

The lifecycle emitted directly by the Agent is intentionally small:

LOGON -> ACTIVE
LOGOFF -> CLOSED

Reconciliation-based and reboot-based closure remain responsibilities of the central API.

This keeps local collection focused on observable facts while consolidated interpretation stays centralized.

Polling and snapshots

Default values in 0.1.0 are:

{
  "poll_seconds": 10,
  "snapshot_seconds": 30,
  "request_timeout_seconds": 10,
  "auth_match_window_seconds": 180,
  "max_batch_events": 100
}

The daemon continuously runs cycles.

Each cycle roughly performs:

1. read boot_id and local state
2. read journald after the saved cursor
3. merge Accepted records and PAM signals
4. enumerate logind sessions
5. resolve new sessions and closures
6. detect complete ephemeral sessions
7. enqueue events/snapshots locally
8. persist state
9. flush the spool in order

Journald cursor

Local state stores a journal_cursor for incremental reads.

When a cursor exists, the Agent uses:

journalctl --after-cursor <cursor>

If the cursor disappeared because of rotation or vacuum, the reader falls back to a bounded lookback rather than attempting to replay unlimited history.

Atomic state persistence

State is stored in:

/var/lib/ssh-session-agent/state.json

It contains:

• boot_id;
• journal cursor;
• known active sessions;
• recent authentication observations;
• last snapshot timestamp.

Writes use a temporary file, fsync, mode 0600, and os.replace.

This reduces the chance that a crash leaves a partially written JSON file.

Spool before checkpoint advancement

Events and snapshots are stored under:

/var/lib/ssh-session-agent/spool/

The durability rule is:

produce telemetry
 -> atomically write spool item
 -> save new state
 -> transmit
 -> remove only after success

This ordering prevents the journal cursor from silently advancing past telemetry that was never queued.

A crash in a small window may cause replay, but API v2 ingestion is idempotent.

Silent loss would be worse than retransmission.

Queue ordering

The spool is flushed in deterministic order.

When one item fails:

current item remains
flush stops
later items wait

This prevents later telemetry from overtaking an earlier failure and keeps recovery behavior predictable.

Dead-letter handling

A corrupt local spool item should not block the entire queue forever.

Items that cannot be decoded or do not contain the required structure are moved to:

/var/lib/ssh-session-agent/dead-letter/

They remain available for investigation while the rest of the queue can continue.

API v2 contract

Events are sent to:

POST /api/v2/agent/events

Using the generic identity model:

{
  "platform": "linux",
  "protocol": "SSH",
  "boot_id": "...",
  "events": [
    {
      "type": "LOGON",
      "provider_session_id": "logind:42",
      "provider_event_id": "logind:42:logon",
      "username": "example.user",
      "source_ip": "192.0.2.20",
      "source_port": 53122
    }
  ]
}

Snapshots use:

POST /api/v2/agent/snapshot

Authentication

Every monitored Linux host receives its own credential:

X-Server-ID: <server-id>
Authorization: Bearer <agent-secret>

The Agent never receives the global query API key used by read consumers.

The HTTP client uses Python’s standard library with certificate verification enabled.

When needed, configuration can reference a dedicated CA bundle without disabling TLS validation.

Installation model

The recommended production checkout is:

/opt/SSH-Session-Agent

Configuration is kept separately in:

/etc/ssh-session-agent/config.json

And mutable state in:

/var/lib/ssh-session-agent

Installation creates a dedicated ssh-session-agent system user with no interactive shell and only the journal access required by the collector.

systemd hardening

The unit includes controls such as:

NoNewPrivileges=true
PrivateTmp=true
PrivateDevices=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictSUIDSGID=true
RestrictNamespaces=true
LockPersonality=true

Write access is explicitly restricted to the state directory, while configuration is exposed read-only to the service.

The daemon does not need to run as root.

Preflight

Before production start, the Agent validates:

• Python 3.10+;
• Debian/Ubuntu compatibility;
• systemctl, journalctl, and loginctl;
• active systemd-logind;
• an available OpenSSH unit;
• read access to journald;
• write access to the state directory;
• the configured CA file, when used;
• API v2 health over HTTPS.

The preflight health request is read-only and does not send the Agent bearer token.

Controlled installation

The installer can enable the unit without immediately starting it.

This supports a workflow like:

install files
 -> review preflight
 -> validate credentials/configuration
 -> start the service

In infrastructure work, separating “files are installed” from “telemetry starts flowing” reduces the blast radius of preparation mistakes.

Upgrade and rollback

Configuration, state, and spool live outside the application checkout.

After moving /opt/SSH-Session-Agent to an approved release or commit:

sudo bash /opt/SSH-Session-Agent/scripts/update.sh

The updater runs preflight as the service account, restarts the unit, and verifies that it becomes active.

For immediate containment:

sudo systemctl stop ssh-session-agent.service

The default uninstall removes the unit but preserves configuration and state. Destructive purge requires explicit flags.

Privacy boundary

The Agent does not collect:

• executed commands;
• stdin/stdout;
• terminal contents;
• passwords;
• private-key material;
• public-key contents or fingerprints;
• clipboard data.

It retains only metadata needed to represent the session and its origin.

That boundary was established in the MVP so access observability would not become user-activity recording.

Validation and rollout

The MVP was designed to begin on one non-critical Linux host before broad deployment.

The gate covers scenarios including:

• bootstrap without false historical LOGONs;
• public-key and password authentication;
• simultaneous sessions;
• normal logout;
• short ephemeral sessions;
• API outage and spool replay;
• reboot behavior;
• no lifecycle duplication;
• no regression in the existing RDP path.

Broader Linux rollout remains gated behind a stable pilot window.

Current state

Version 0.1.0 provides a functional Debian/Ubuntu SSH collector with:

• logind as the authority for persistent sessions;
• journald/OpenSSH enrichment;
• PAM signals;
• ephemeral-session recovery;
• kernel boot_id;
• atomic state and spool persistence;
• idempotent replay;
• periodic snapshots;
• validated TLS;
• a dedicated service account;
• systemd hardening;
• Remote Session API v2 integration.

What the project demonstrates

SSH Session Agent is not just a log parser.

The project demonstrates how to combine multiple evidence sources with different confidence levels, prevent lifecycle duplication, recover short-lived activity, preserve telemetry through failures, and define a strict privacy boundary.

It also represents the most important architectural shift in the monitoring stack: moving from an RDP-specific solution to a common remote-session domain while keeping each collector specialized for the operating system it observes.