---
title: "Let Pioneer AI control a session"
description: "Approve one task at a time for Pioneer AI to see and operate an endpoint's screen in a remote session you have open, and watch or stop it."
url: "https://support.outpostplatform.com/remote-access/pioneer-control/"
product: "depot"
type: "how-to"
status: "draft"
ai_summary: "Pioneer AI operates the screen of a remote session the technician has open for one approved task (never auto-approved), from the viewer's Pioneer AI drawer or the console chat. Default budget 40 actions, 10 minutes. Stops on Stop, technician input, Windows secure screens, or budget. Needs a vision model, remote_access.ai_control, and remote:ai_control. Screenshots are not stored."
availability:
  plans: ["home-lab", "essentials", "enterprise", "msp"]
  platforms: ["windows", "linux", "macos"]
  permission: "remote:ai_control"
  preview: false
source: "https://gitlab.com/outpostplatform/docs/-/edit/main/src/content/docs/remote-access/pioneer-control.md"
license: "CC BY 4.0"
---

# Let Pioneer AI control a session

During a remote desktop session, Pioneer AI can operate the endpoint's screen
for you: it reads the screen and uses the mouse and keyboard to carry out one
task you approve, such as finding a setting in a control panel or reading
values from an application that has no command line. You watch every step in
your viewer and can take over at any moment. This page shows how to start a
task, follow it, and stop it, and what to check when the option is not
available.

## How it works

You ask Pioneer AI for something that needs the screen. It asks you to approve
one task: the endpoint, the goal, and a budget of actions and minutes. Once you
approve, Pioneer AI works in steps. At each step it takes a screenshot of the
picture in your viewer, sends it to the AI model, and performs one action: a
click, a drag, typed text, a key combination, a scroll, or a wait. Its pointer
glides to each target before it clicks, so you can follow what it is about to
do. When the task is done, or something stops it, Pioneer AI reports what it
did and what it found in the conversation.

Pioneer AI prefers commands, scripts, and other tools when they can do the job,
and uses the screen for work only a person at the desktop could otherwise do.
When it works in a shell, the viewer's terminal shows the shell as it runs. See
[Work with Pioneer AI in the terminal](/remote-access/terminal/#pioneer-ai).

## Before you begin

- **Permissions.** You need `remote:initiate` and the permission for the
  session's mode, `ai:use`, and `remote:ai_control`
  (**Let Pioneer AI control remote sessions**). Every role that can start
  remote sessions holds `remote:ai_control` by default. See
  [Permissions](/remote-access/#permissions).
- **Policy.** The endpoint's `remote_access.ai_control` profile setting
  (**Allow Pioneer AI to control remote sessions** in the profile editor) must
  be on. It is on unless a profile turns it off. See
  [Remote access policy settings](/remote-access/policy-settings/#remote-access).
- **A vision model.** Pioneer AI needs a model that can read screenshots. With
  Outpost Cloud AI, Outpost provides it. With your own endpoint, an owner
  enters one in **Vision model (optional)** on **Settings** > **AI**. See
  [Use your own endpoint](/pioneer-ai/set-up/#use-your-own-endpoint).
- **An open session.** Pioneer AI acts only on a session you have open in the
  remote viewer, in the same browser as the conversation. See
  [Start a remote desktop session](/remote-access/start-a-session/).
- **What is sent.** Each step sends a screenshot of the session to the AI
  provider your organization uses. Close or hide anything on the endpoint's
  screen that must not leave it before you approve a task. See
  [Screenshots of a remote session](/pioneer-ai/data-and-privacy/#screenshots-of-a-remote-session).

:::caution
Pioneer AI acts as the user signed in at the endpoint, with that user's
rights. It is told never to type passwords or other secrets and never to
change security settings unless the goal says so, but it can still click the
wrong thing. Watch the task, and take over if it goes somewhere you did not
intend.
:::

## Start a task from the viewer

1. In the remote viewer's toolbar, select **Pioneer AI**. A Pioneer AI drawer
   opens beside the endpoint's screen. If a dialog opens instead, see
   [If Pioneer AI control is not available](#not-available).
2. Describe the task, for example "Open Device Manager and tell me which
   devices show a warning". The drawer already knows which endpoint you are
   connected to.
3. Approve the task. See [Approve the task](#approve).

Drag the drawer's edge to resize it. Closing the drawer does not stop a task
that is running.

The drawer starts a new conversation each time the viewer window connects to a
session. Reopening it during the same session keeps the conversation, and
earlier ones stay in your conversation history.

## Start a task from the console chat

Pioneer AI in the console's drawer, or in a conversation opened in its own
window, can also control a session.

1. Open the endpoint's session in the remote viewer, in the same browser.
2. In the conversation, ask for the task and name the endpoint.
3. Approve the task.

If no viewer for that endpoint is open, Pioneer AI says so and can offer a
**Connect** card to start the session. Connect, then ask again.

## Approve the task {#approve}

Pioneer AI shows the usual approval card, **Pioneer AI wants to take an
action**, with the **High impact** tier. The card names the task, for
example:

> Control PC-1's screen: Open Device Manager and list devices with a warning (up to 40 actions, 10 min)

Select **Approve** to start, or **Deny** to refuse. One approval covers the
whole task and nothing after it. A new task needs a new approval.

[Automatic approval](/pioneer-ai/automatic-approval/) never approves a
remote control task, whatever mode an owner chose. A person always decides.

Each task has a budget. Pioneer AI asks for 40 actions and 10 minutes unless
you ask for more or less, up to 60 actions and 20 minutes. Every step counts
as an action, including waits and retries.

## Follow the task

While Pioneer AI is in control:

- The status bar reads **Pioneer AI is controlling this session. Step 3 of
  40.**, with **Stop**.
- A **Pioneer** badge appears beside the endpoint's name.
- Pioneer AI's own pointer shows where it is about to act.
- The conversation shows a card, **Controlling** and the endpoint's name, with
  **Running**, the goal, **Step 3 of 40**, the last action (such as
  `Click (412, 233)` or `Press Ctrl+S`), the time left, and **Stop**.

The person at the endpoint sees the pointer move and text appear, as they
would for a technician. They see no extra prompt.

## Stop or take over

Any of these stops the task at once:

- **Stop** in the status bar or on the card.
- A click, a key press, or a scroll of your own in the viewer. Pioneer AI
  hands control back to you. Moving the mouse alone does not count.
- A sign-in, UAC, or lock screen appearing on a Windows endpoint.
- The task using its last action or minute.
- The session ending.

When a task ends, the card shows **Ended** and the reason, and Pioneer AI
reports what it did in the conversation:

| The card says | What happened |
| --- | --- |
| **Pioneer AI finished the task.** | The task is done. The answer follows in the conversation. |
| **You took control, so Pioneer AI stopped.** | You clicked, pressed a key, or scrolled in the viewer. |
| **Stopped.** | Someone selected **Stop**. |
| **The task used its 40 actions.** | The action budget ran out. Ask again to continue. |
| **The task ran out of time.** | The time budget ran out. Ask again to continue. |
| **A sign-in, UAC or lock screen appeared, so Pioneer AI stopped. Handle it in the viewer.** | Windows switched to a secure screen. Deal with it yourself, then ask Pioneer AI to carry on. |
| **The reply reached its spending limit.** | The reply reached the [spending limit per reply](/pioneer-ai/use-pioneer-ai/#spending-limit-per-reply). |
| **The remote viewer for this endpoint is not open.** | No viewer for the endpoint was open in this browser. |
| **The remote viewer stopped answering.** | The viewer did not return a screenshot within 20 seconds, for example because it was closed. |
| **The remote session ended.** | The session ended or paused. |
| **Pioneer AI control is no longer allowed for this session.** | The policy, your permissions, or the AI settings changed during the task. |

:::note
macOS and Linux endpoints do not report sign-in or lock screens to the viewer.
On those platforms Pioneer AI stops at one only when it recognizes it in the
screenshot, so watch for them.
:::

## If Pioneer AI control is not available {#not-available}

When Pioneer AI cannot control the session, **Pioneer AI** in the viewer opens
**Pioneer AI control is not available**, which explains the feature and gives
the reason. In the console chat, Pioneer AI gives the same reason instead of
asking for approval.

| Reason shown | What to do |
| --- | --- |
| **AI features are turned off for your organization.** | Ask someone with `ai:manage` to turn Pioneer AI on. See [Set up Pioneer AI](/pioneer-ai/set-up/). |
| **AI features are turned off for this client.** | On the MSP plan, ask someone with `ai:manage` to change the client's AI access. See [How MSPs control it per client](/pioneer-ai/#how-msps-control-it-per-client). |
| **No AI model that can read screens is set up. An owner can choose one in Settings > AI.** | With your own endpoint, an owner enters a model in **Vision model (optional)**. See [Use your own endpoint](/pioneer-ai/set-up/#use-your-own-endpoint). |
| **This endpoint's remote access policy does not allow Pioneer AI to control sessions.** | A profile that applies to the endpoint turns off `remote_access.ai_control`. Change the profile, or work without Pioneer AI. See [Remote access policy settings](/remote-access/policy-settings/#remote-access). |
| **Your role does not include permission to let Pioneer AI control sessions.** | Your role lacks `remote:ai_control` or `ai:use`. Ask someone with `role:manage` to add it. See [Roles and permissions](/depot/roles-and-permissions/). |
| **This endpoint's agent is too old for Pioneer AI control. Update the agent to use it.** | Update the Outpost agent on the endpoint. This reason appears once the session is connected. |

## What is recorded

- The conversation keeps the approval, each step as text, and Pioneer AI's
  report. It never keeps the screenshots.
- The [audit log](/depot/audit-log/) records the start and end of each task
  against the remote session, as `remote.ai_drive.start` (with the goal and
  budget) and `remote.ai_drive.end` (with the number of actions and the
  reason). Single actions are not audited.
- If the session is recorded, the recording shows Pioneer AI's actions as it
  shows yours. See [Record remote sessions](/remote-access/session-recording/).
- Model calls for a task appear as **Remote control** on **Settings** >
  **AI** > **Usage**. See [Review Pioneer AI usage](/pioneer-ai/usage/).

## Verify

After you approve a task, the status bar reads **Pioneer AI is controlling
this session**, and the step count rises as Pioneer AI works.

## Next steps

- [Approve Pioneer AI actions](/pioneer-ai/approve-actions/)
- [Pioneer AI data and privacy](/pioneer-ai/data-and-privacy/)
- [Remote access policy settings](/remote-access/policy-settings/)