Let Pioneer AI control a session

During a remote desktop session, Pioneer AI can operate the endpoint’s screen for you: it reads the screen and uses the mouse and keyboard to carry out one task you approve, such as finding a setting in a control panel or reading values from an application that has no command line. You watch every step in your viewer and can take over at any moment. This page shows how to start a task, follow it, and stop it, and what to check when the option is not available.

How it works

You ask Pioneer AI for something that needs the screen. It asks you to approve one task: the endpoint, the goal, and a budget of actions and minutes. Once you approve, Pioneer AI works in steps. At each step it takes a screenshot of the picture in your viewer, sends it to the AI model, and performs one action: a click, a drag, typed text, a key combination, a scroll, or a wait. Its pointer glides to each target before it clicks, so you can follow what it is about to do. When the task is done, or something stops it, Pioneer AI reports what it did and what it found in the conversation.

Pioneer AI prefers commands, scripts, and other tools when they can do the job, and uses the screen for work only a person at the desktop could otherwise do. When it works in a shell, the viewer’s terminal shows the shell as it runs. See Work with Pioneer AI in the terminal.

Before you begin

• Permissions. You need remote:initiate and the permission for the session’s mode, ai:use, and remote:ai_control (Let Pioneer AI control remote sessions). Every role that can start remote sessions holds remote:ai_control by default. See Permissions.
• Policy. The endpoint’s remote_access.ai_control profile setting (Allow Pioneer AI to control remote sessions in the profile editor) must be on. It is on unless a profile turns it off. See Remote access policy settings.
• A vision model. Pioneer AI needs a model that can read screenshots. With Outpost Cloud AI, Outpost provides it. With your own endpoint, an owner enters one in Vision model (optional) on Settings > AI. See Use your own endpoint.
• An open session. Pioneer AI acts only on a session you have open in the remote viewer, in the same browser as the conversation. See Start a remote desktop session.
• What is sent. Each step sends a screenshot of the session to the AI provider your organization uses. Close or hide anything on the endpoint’s screen that must not leave it before you approve a task. See Screenshots of a remote session.

Caution

Pioneer AI acts as the user signed in at the endpoint, with that user’s rights. It is told never to type passwords or other secrets and never to change security settings unless the goal says so, but it can still click the wrong thing. Watch the task, and take over if it goes somewhere you did not intend.

Start a task from the viewer

1. In the remote viewer’s toolbar, select Pioneer AI. A Pioneer AI drawer opens beside the endpoint’s screen. If a dialog opens instead, see If Pioneer AI control is not available.
2. Describe the task, for example “Open Device Manager and tell me which devices show a warning”. The drawer already knows which endpoint you are connected to.
3. Approve the task. See Approve the task.

Drag the drawer’s edge to resize it. Closing the drawer does not stop a task that is running.

The drawer starts a new conversation each time the viewer window connects to a session. Reopening it during the same session keeps the conversation, and earlier ones stay in your conversation history.

Start a task from the console chat

Pioneer AI in the console’s drawer, or in a conversation opened in its own window, can also control a session.

1. Open the endpoint’s session in the remote viewer, in the same browser.
2. In the conversation, ask for the task and name the endpoint.
3. Approve the task.

If no viewer for that endpoint is open, Pioneer AI says so and can offer a Connect card to start the session. Connect, then ask again.

Approve the task

Pioneer AI shows the usual approval card, Pioneer AI wants to take an action, with the High impact tier. The card names the task, for example:

Control PC-1’s screen: Open Device Manager and list devices with a warning (up to 40 actions, 10 min)

Select Approve to start, or Deny to refuse. One approval covers the whole task and nothing after it. A new task needs a new approval.

Automatic approval never approves a remote control task, whatever mode an owner chose. A person always decides.

Each task has a budget. Pioneer AI asks for 40 actions and 10 minutes unless you ask for more or less, up to 60 actions and 20 minutes. Every step counts as an action, including waits and retries.

Follow the task

While Pioneer AI is in control:

• The status bar reads Pioneer AI is controlling this session. Step 3 of 40., with Stop.
• A Pioneer badge appears beside the endpoint’s name.
• Pioneer AI’s own pointer shows where it is about to act.
• The conversation shows a card, Controlling and the endpoint’s name, with Running, the goal, Step 3 of 40, the last action (such as Click (412, 233) or Press Ctrl+S), the time left, and Stop.

The person at the endpoint sees the pointer move and text appear, as they would for a technician. They see no extra prompt.

Stop or take over

Any of these stops the task at once:

• Stop in the status bar or on the card.
• A click, a key press, or a scroll of your own in the viewer. Pioneer AI hands control back to you. Moving the mouse alone does not count.
• A sign-in, UAC, or lock screen appearing on a Windows endpoint.
• The task using its last action or minute.
• The session ending.

When a task ends, the card shows Ended and the reason, and Pioneer AI reports what it did in the conversation:

The card says	What happened
Pioneer AI finished the task.	The task is done. The answer follows in the conversation.
You took control, so Pioneer AI stopped.	You clicked, pressed a key, or scrolled in the viewer.
Stopped.	Someone selected Stop.
The task used its 40 actions.	The action budget ran out. Ask again to continue.
The task ran out of time.	The time budget ran out. Ask again to continue.
A sign-in, UAC or lock screen appeared, so Pioneer AI stopped. Handle it in the viewer.	Windows switched to a secure screen. Deal with it yourself, then ask Pioneer AI to carry on.
The reply reached its spending limit.	The reply reached the spending limit per reply.
The remote viewer for this endpoint is not open.	No viewer for the endpoint was open in this browser.
The remote viewer stopped answering.	The viewer did not return a screenshot within 20 seconds, for example because it was closed.
The remote session ended.	The session ended or paused.
Pioneer AI control is no longer allowed for this session.	The policy, your permissions, or the AI settings changed during the task.

Note

macOS and Linux endpoints do not report sign-in or lock screens to the viewer. On those platforms Pioneer AI stops at one only when it recognizes it in the screenshot, so watch for them.

If Pioneer AI control is not available

When Pioneer AI cannot control the session, Pioneer AI in the viewer opens Pioneer AI control is not available, which explains the feature and gives the reason. In the console chat, Pioneer AI gives the same reason instead of asking for approval.

Reason shown	What to do
AI features are turned off for your organization.	Ask someone with ai:manage to turn Pioneer AI on. See Set up Pioneer AI.
AI features are turned off for this client.	On the MSP plan, ask someone with ai:manage to change the client’s AI access. See How MSPs control it per client.
No AI model that can read screens is set up. An owner can choose one in Settings > AI.	With your own endpoint, an owner enters a model in Vision model (optional). See Use your own endpoint.
This endpoint’s remote access policy does not allow Pioneer AI to control sessions.	A profile that applies to the endpoint turns off remote_access.ai_control. Change the profile, or work without Pioneer AI. See Remote access policy settings.
Your role does not include permission to let Pioneer AI control sessions.	Your role lacks remote:ai_control or ai:use. Ask someone with role:manage to add it. See Roles and permissions.
This endpoint’s agent is too old for Pioneer AI control. Update the agent to use it.	Update the Outpost agent on the endpoint. This reason appears once the session is connected.

What is recorded

• The conversation keeps the approval, each step as text, and Pioneer AI’s report. It never keeps the screenshots.
• The audit log records the start and end of each task against the remote session, as remote.ai_drive.start (with the goal and budget) and remote.ai_drive.end (with the number of actions and the reason). Single actions are not audited.
• If the session is recorded, the recording shows Pioneer AI’s actions as it shows yours. See Record remote sessions.
• Model calls for a task appear as Remote control on Settings > AI > Usage. See Review Pioneer AI usage.

Verify

After you approve a task, the status bar reads Pioneer AI is controlling this session, and the step count rises as Pioneer AI works.

Next steps

• Approve Pioneer AI actions
• Pioneer AI data and privacy
• Remote access policy settings