LLDB Remote Debugging

Remote debugging is a critical capability that enables developers to debug code on a target device without running the debugger directly on that device. This guide demonstrates how to utilize lldb and lldb-server to implement effective remote debugging workflows.

Why Remote Debugging?

In scenarios such as embedded systems development and remote server maintenance, the target device often has limited resources or lacks a complete development environment. Remote debugging allows developers to run the heavy-lifting debugger on a local machine while connecting to a lightweight debug server on the remote device, ensuring an efficient and minimally intrusive debugging process.

What are LLDB and lldb-server?

LLDB is the debugger for the LLVM project, supporting multiple programming languages and platforms. lldb-server is a component of LLDB that functions as a remote debugging server, allowing LLDB clients to connect to and control remote processes. The relationship between LLDB and lldb-server parallels that of GDB and gdbserver. However, LLDB offers superior platform-specific capabilities. Unlike GDB, which delegates most platform interaction to gdbserver, LLDB’s SBPlatform abstraction object provides robust management of remote file systems, processes, and environments. This highlights the architectural advantages of LLDB:

1. Remote File and Path Mapping: LLDB’s SBPlatform natively understands how to access the target machine’s file system via lldb-server. This is critical for handling Sysroots, shared libraries, symbol file discovery, and source code path mapping. GDB relies more heavily on local files or manual configuration. For instance, in LLDB, a Module’s path serves merely as an identifier; its reachability and access method are semantically defined entirely by the Platform.
2. Granular Remote Process Control: SBPlatform is not solely responsible for launching processes; it manages the entire remote environment setup, including environment variables and working directories.
3. Source of Truth: Remote (LLDB) vs. Local (GDB): LLDB leverages SBPlatform to collaborate with lldb-server, intelligently locating dependent system libraries (Sysroot) on the remote target and loading their corresponding symbols onto the host.
4. Extensibility: The platform layer is highly extensible, allowing developers to write custom platform adapters for new operating systems or embedded environments.

1. Architecture Comparison

2. Remote Debugging Capabilities Comparison

Although LLDB is more modern and powerful than GDB, GDB still holds advantages when debugging bare metal and certain legacy platforms. Choosing the right tool is key to efficiency.

LLDB Command Line Debugging

1. Starting lldb-server on the Target Device

To initiate remote debugging, lldb-server must first be started on the device. Use the following command to start lldb-server in Platform mode:

lldb-server platform --server --listen "*:1234"

2. Connecting to lldb-server from the Local Machine

Launch lldb on the local machine and connect to the remote lldb-server:

lldb
(lldb) platform select remote-linux
(lldb) platform connect connect://<remote-ip>:1234

Upon a successful connection, information about the current remote platform will be displayed, for example:

(lldb) platform connect connect://192.168.139.113:1234
  Platform: remote-linux
    Triple: aarch64-unknown-linux-gnu
OS Version: 6.17.8 (6.17.8-orbstack-00308-g8f9c941121b1)
  Hostname: ubuntu-arm
 Connected: yes
WorkingDir: /home/hidka
    Kernel: #1 SMP PREEMPT Thu Nov 11 09:34:02 UTC 2025

Once connected, you can utilize various LLDB debugging commands. Notably, in Platform mode, you only need to use target create <local_executable_path> locally to load the target. Upon executing the run command, LLDB automatically uploads the executable to the remote device. If the target program already exists on the remote device, you can specify its location via the --remote-file <remote_path> parameter when creating the target, or directly use process attach to attach to a running remote process.

VSCode Configuration for LLDB Remote Debugging

1. Install lldb-dap Extension

The LLVM project offers Debug Adapter Protocol (DAP) support for VSCode. Install the lldb-dap extension from the VSCode Marketplace and ensure the lldb-dap binary (usually provided with the LLVM toolchain) is installed.

2. Configure launch.json

Add the remote debugging configuration to the VSCode debugging configuration file launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "lldb-dap",
            "request": "launch",
            "name": "Debug",
            "program": "${workspaceFolder}/out/build/Debug/your_executable",
            "initCommands": [
                "platform select remote-linux",
                "platform connect connect://<remote-ip>:1234"
            ]
        }
    ]
}

Once configured, initiate the remote debugging session by clicking the debug button in VSCode. The official documentation for the lldb-dap extension details the meaning and usage of each parameter; referring to it for advanced configuration is recommended.

Common Issues and Solutions

1. Installation

• Linux: It is recommended to use the official llvm.sh script to install the latest version of the LLDB toolkit and Clang toolchain.
• Windows: Install via the official LLVM installer.
• Android: Utilize the Google-provided SDK toolchain, which includes lldb-server for Android.
• Embedded Devices: For embedded devices, consider cross-compiling lldb-server for the target architecture.

2. Troubleshooting Connection Failures

Network and Firewall (Standard Linux Devices)

• Verify Port Accessibility: Ensure the target device’s firewall (e.g., iptables, ufw) allows inbound traffic on the port lldb-server is listening on (default 1234).
• Connectivity Testing: Use telnet <ip> <port> or nc -zv <ip> <port> on the host machine to test connectivity. You can also use nmap -p <port> <ip> to scan the target port status.

Android Connection Mechanisms

Android debugging usually relies on ADB communication rather than direct TCP connections.

• Platform Selection: Use platform select remote-android.
• Connection Method:
  • Unix Abstract Socket (Recommended): Start lldb-server on the device listening on a Unix Abstract Socket, and connect from the host using platform connect unix-abstract-connect://<socket_name>. In this case, LLDB will forward data internally via the ADB protocol without manually setting up port forwarding.
  • TCP Port: If listening on a TCP port, you can connect directly using connect://localhost:<local> because LLDB will automatically execute adb forward.

Restricted Environments (Air-gapped/Closed Ports)

For devices that cannot be accessed directly via the network (e.g., high-security devices with only USB or serial connections):

• Port Forwarding: Establish a data channel between the host and the device. You can map the device’s debug interface to a local port (Localhost) on the host machine by writing tools or using existing tools (e.g., SSH Tunnel, USB mapping tools), and then connect LLDB to the local port.
• Serial Communication: If the device only has a serial connection, use tools like socat to forward serial data to a local TCP port, and then connect LLDB to that port.

3. Version Mismatches

Significant version discrepancies between the LLDB client on the host and lldb-server on the target may lead to communication protocol incompatibilities, resulting in inexplicable packet error messages or connection interruptions.

• Recommendation: Strive to keep the versions on both ends consistent. If complete consistency is not possible, ensure Host Version >= Target Version.

4. Missing Symbols (Source Code/Stack Trace Unavailable)

If the connection is successful but bt only shows memory addresses, or source code cannot be viewed, it is usually because the host lacks the symbol files or system library copies (Sysroot) of the target machine.

• Set Sysroot:

  (lldb) platform select remote-linux --sysroot /path/to/local/copy/of/target/rootfs
  

• Manually Load Symbols:

  (lldb) target modules add /path/to/local/symbol/file
  

5. Insufficient Permissions (Attach Failure)

When attaching to a running process on Linux/Android, you may encounter an Operation not permitted error. This is due to the kernel’s ptrace_scope security restriction.

• Solution:
  Execute on the target machine:

  echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
  

  Or run lldb-server directly with root privileges.

Conclusion

Remote debugging is a powerful tool that can greatly improve development and debugging efficiency. By flexibly using lldb and lldb-server, you can successfully carry out remote debugging work in various complex environments.