ECMAScript debugger protocol

Origins

The current XML protocol is based on the old "version 3" binary protocol with some structuring. See the end of this document for a critique.

Versioning

This document describes version 3 of the protocol. This number is sent as the PROTOCOL-VERSION in the HELLO message.

Flow

Control flow is on the pattern EVENT (COMMAND RESPONSE?)* COMMAND: Opera (aka "the debugging host") sends an event to the debugger which may issue commands to Opera (some of which have responses), and finally issues a command to relinquish control until the next event. The only command that relinquishes control is "continue".

The initial event is "hello" which is sent once Opera discovers that a debugger is present.

There is one exception to this scheme. While the debugger is not in control it may send a "break" command to Opera to force it to stop execution, and signal breakpoint events for all threads.

Central data types and how they work

A "runtime" represents a document context in which threads execute. There is a one-to-one mapping between runtimes and HTML documents.

A "thread" represents a thread of execution in a runtime. A "parent" thread can be preempted by another "child" thread in order to respond to a priority event; the parent thread will not run again until the child thread has finished.

A "script" is a pair (id, source code) where the id is globally unique and the source code is Unicode text.

An "object" is represented by a globally unique ID that is assigned by the debugger.

Globally unique means unique within a running Opera session; that is from starting Opera to stopping it.

Protocol principles

The main principle is that the data should be self-describing. This means that even enumerated values are passed as strings and that element names are verbose.

BNF

Here is the grammar describing the data. Some of the data elements are described in more detail below.

  ###
  # The protocol can mostly be described as a context-free grammar of
  # data flowing past an observer on the wire, where HELLO and EVENT
  # flows from host to client, and COMMAND flows from client to HOST.
  #

  PROTOCOL ::= HELLO (COMMAND | EVENT)* ;

  ###
  # Events (messages from debugging host to debugger)
  #

  EVENT ::= RUNTIME-STARTED
          | RUNTIME-STOPPED
          | NEW-SCRIPT
          | PARSE-ERROR
          | THREAD-STARTED
          | THREAD-FINISHED
          | THREAD-STOPPED-AT
          | EVAL-REPLY
          | EXAMINE-REPLY
          | BACKTRACE-REPLY
          | RUNTIMES-REPLY
          | HANDLE-EVENT
          | OBJECT-SELECTED
          ;

  HELLO ::= "<hello>"
              PROTOCOL-VERSION      # from debugging host
              OPERATING-SYSTEM
              PLATFORM
              USER-AGENT            # the one that's statically configured
            "</hello>" ;

  RUNTIME-STARTED ::= "<runtime-started>" RUNTIME "</runtime-started>" ;

  RUNTIME-STOPPED ::= "<runtime-stopped>" RUNTIME-ID "</runtime-stopped>" ;

  NEW-SCRIPT ::= "<new-script>"
                   RUNTIME-ID
                   SCRIPT-ID
                   SCRIPT-TYPE
                   SCRIPT-DATA
                   URI?             # present if SCRIPT-TYPE is "linked"
                 "</new-script>" ;

  # OFFSET represents the character offset in the script where the parse error occured

  OFFSET ::= "<offset>" UNSIGNED "</offset>" ;

  # CONTEXT describes in what context the error occured

  CONTEXT ::= "<context>" TEXT "</context>" ;

  # DESCRIPTION contains the human-readable description of the parse error

  DESCRIPTION ::= "<description>" TEXT "</description>" ;

  PARSE-ERROR ::= "<parse-error>"
                     RUNTIME-ID
                     SCRIPT-ID
                     LINE-NUMBER
                     OFFSET
                     CONTEXT
                     DESCRIPTION
                   "</parse-error>" ;

  THREAD-STARTED ::= "<thread-started>"
                       RUNTIME-ID
                       THREAD-ID
                       PARENT-THREAD-ID
                       THREAD-TYPE
                       EVENT-DESC?  # present if THREAD-TYPE is "event"
                     "</thread-started>" ;

  THREAD-FINISHED ::= "<thread-finished>"
                        RUNTIME-ID
                        THREAD-ID
                        STATUS
                      "</thread-finished>" ;

  # BREAKPOINT-ID is present if and only if STOPPED-REASON is "breakpoint"

  THREAD-STOPPED-AT ::= "<thread-stopped-at>"
                          RUNTIME-ID
                          THREAD-ID
                          SCRIPT-ID
                          LINE-NUMBER
                          STOPPED-REASON
                          BREAKPOINT-ID?
                        "</thread-stopped-at>" ;

  # If STATUS is "completed" or "unhandled-exception", then
  # VALUE-DATA will be present.

  EVAL-REPLY ::= "<eval-reply>"
                   TAG
                   STATUS
                   VALUE-DATA?
                 "</eval-reply>" ;

  EXAMINE-REPLY ::= "<examine-reply>"
                      TAG
                      OBJECT*
                    "</examine-reply>" ;

  # Frames are in innermost-first order

  BACKTRACE-REPLY ::= "<backtrace-reply>"
                        TAG
                        FRAME*
                      "</backtrace-reply>" ;

  RUNTIMES-REPLY ::= "<runtimes-reply>"
                       TAG
                       RUNTIME*
                     "</runtimes-reply>" ;

  # This event is issued for XML events on the host, if a corresponding
  # ADD-EVENT-HANDLER has been issued earlier by the client.
  # OBJECT-ID refers to the target of the event.

  HANDLE-EVENT ::= "<handle-event>"
                     OBJECT-ID
                     HANDLER-ID
                     EVENT-TYPE
                   "</handle-event>" ;

  # Some hosts send this event to indicate that an object was selected for
  # debugging, e.g., if the debugger was started by right-clicking an element
  # and clicking "inspect" in the context menu, this event will be sent
  # right after startup. A client may safely choose to ignore this event.

  OBJECT-SELECTED ::= "<object-selected>"
                        OBJECT-ID
                        WINDOW-ID
                      "</object-selected>" ;

  ###
  # Commands (messages from debugger to debugging host)
  #

  COMMAND ::= RUNTIMES
            | CONTINUE
            | EVAL
            | EXAMINE-FRAME
            | EXAMINE-OBJECTS
            | SPOTLIGHT-OBJECT
            | ADD-BREAKPOINT
            | REMOVE-BREAKPOINT
            | ADD-EVENT-HANDLER
            | REMOVE-EVENT-HANDLER
            | SET-CONFIGURATION
            | BACKTRACE
            | BREAK
            ;

  RUNTIMES ::= "<runtimes>"
                 TAG
                 CREATE-ALL-RUNTIMES?
                 RUNTIME-ID*        # list the ones you want to see, or none if you want all
               "</runtimes>" ;

  # Create runtimes for all documents. Runtimes are normally not created for documents
  # without ECMAScript.

  CREATE-ALL-RUNTIMES ::= "<create-all-runtimes />" ;

  CONTINUE ::= "<continue>"
                 RUNTIME-ID
                 THREAD-ID
                 MODE
               "</continue>" ;

  # SCRIPT-DATA represents a script to be executed; PROPERTY values
  # represent variables to set.
  # If THREAD-ID, code is evaluated in the global scope.

  EVAL ::= "<eval>"
             TAG
             RUNTIME-ID
             THREAD-ID
             FRAME-ID
             SCRIPT-DATA
             PROPERTY*
           "</eval>" ;

  EXAMINE-FRAME ::= "<examine-frame>"
                      TAG
                      RUNTIME-ID
                      THREAD-ID
                      FRAME-ID
                    "</examine-frame>" ;

  EXAMINE-OBJECTS ::= "<examine-objects>"
                        TAG
                        RUNTIME-ID
                        OBJECT-ID+
                      "</examine-objects>" ;

  # Using OBJECT-ID == 0 clears the spotlight.
  # If SCROLL-INTO-VIEW is present, the object will be scrolled into the view (at least part of it),
  # otherwise the viewport will remain where it is.

  SPOTLIGHT-OBJECT ::= "<spotlight-object>"
                         OBJECT-ID
                         SCROLL-INTO-VIEW?
                       "</spotlight-object>" ;

  SCROLL-INTO-VIEW ::= "<scroll-into-view />" ;

  # The SOURCE-POSITION element defines how
  # to set the breakpoint.

  ADD-BREAKPOINT ::= "<add-breakpoint>"
                       BREAKPOINT-ID
                       SOURCE-POSITION
                     "</add-breakpoint>" ;

  REMOVE-BREAKPOINT ::= "<remove-breakpoint>" BREAKPOINT-ID "</remove-breakpoint>" ;

  # Add an event handler. This will generate a HANDLE-EVENT event every time the XML event defined
  # by the pair (NAMESPACE, EVENT-TYPE) reaches the object defined by OBJECT-ID in the capturing
  # phase. XML events are defined in http://www.w3.org/TR/xml-events
  #
  # HANDLER-ID is set by the client and is referred to by both client and host.
  # NAMESPACE of the event: if empty, it will match any namespace.
  # PREVENT-DEFAULT prevents the default event handler from running.
  # STOP-PROPAGATION stops propagation of the event beyond this OBJECT-ID (it will however run for
  # all handlers on the object).

  ADD-EVENT-HANDLER ::= "<add-event-handler>"
                         HANDLER-ID
                         OBJECT-ID
                         NAMESPACE
                         EVENT-TYPE
                         PREVENT-DEFAULT
                         STOP-PROPAGATION
                        "</add-event-handler>" ;

  REMOVE-EVENT-HANDLER ::= "<remove-event-handler>" HANDLER-ID "</remove-event-handler>" ;

  SET-CONFIGURATION ::= "<set-configuration>" STOP-AT+ "</set-configuration>" ;

  # If MAXFRAMES is omitted, all frames are returned.

  BACKTRACE ::= "<backtrace>"
                  TAG
                  RUNTIME-ID
                  THREAD-ID
                  MAXFRAMES?
                "</backtrace>" ;

  BREAK ::= "<break>"
              RUNTIME-ID
              THREAD-ID
            "</break>" ;

  ###
  # Basis-data
  #

  EVENT-DESC ::= "<event-desc>"
                   NAMESPACE
                   EVENT-TYPE
                 "</event-desc>" ;

  PREVENT-DEFAULT ::= "<prevent-default>"
                         YESNO                       # default is yes
                      "</prevent-default>" ;

  STOP-PROPAGATION ::= "<stop-propagation>"
                         YESNO                       # default is yes
                       "</stop-propagation>" ;

  # If DATA-TYPE is ... then ... is present:
  #   "object", OBJECT-ID
  #   "number", STRING
  #   "string", STRING
  #   "boolean", STRING ("true" or "false")
  # Otherwise ("undefined" or "null"), only DATA-TYPE is present.

  VALUE-DATA ::= "<value-data>"
                   DATA-TYPE
                   ( OBJECT-VALUE | STRING )?
                 "</value-data>" ;

  OBJECT-VALUE ::= "<object-value>"
                     OBJECT-ID
                     PROTOTYPE-ID?
                     OBJECT-ATTRIBUTES
                     NAME?
                   "</object-value>" ;

  RUNTIME ::= "<runtime>"
                RUNTIME-ID
                HTML-FRAME-PATH
                WINDOW-ID        # the ID of the window
                OBJECT-ID        # the 'global' object
                URI              # the document's URI
              "</runtime>" ;

  FRAME ::= "<frame>"
              FUNCTION-ID
              ARGUMENT-OBJECT
              VARIABLE-OBJECT
              THIS-OBJECT
              SOURCE-POSITION?
              OBJECT-VALUE*
            "</frame>" ;

  # Default values are NO for every STOP-TYPE, except
  # "script", which is YES.

  STOP-AT ::= "<stop-at>"
                YESNO
                STOP-TYPE
              "</stop-at>" ;

  # Set this value to 0 to get all frames.

  MAXFRAMES ::= "<maxframes>" UNSIGNED "</maxframes>" ;

  SOURCE-POSITION ::= "<source-position>"
                        SCRIPT-ID
                        LINE-NUMBER
                      "</source-position>" ;

  SCRIPT-DATA ::= "<script-data>" TEXT "</script-data>" ;

  OBJECT ::= "<object>"
               OBJECT-VALUE
               PROPERTY*
             "</object>" ;

  PROPERTY ::= "<property>"
                 OBJECT-ID?         # if you want to set a property on an object
                 PROPERTY-NAME
                 VALUE-DATA
               "</property>" ;

  PROPERTY-NAME ::= "<property-name>" TEXT "</property-name>" ;

  OBJECT-ATTRIBUTES ::= "<object-attributes>" OBJECT-ATTRIBUTE* "</object-attributes>" ;

  OBJECT-ATTRIBUTE ::= "<iscallable/>" | "<isfunction/>" ;

  NAME ::= CLASS-NAME | FUNCTION-NAME ;

  CLASS-NAME ::= "<class-name>" TEXT "</class-name>" ;

  FUNCTION-NAME ::= "<function-name>" TEXT "</function-name>" ;

  PROTOCOL-VERSION ::= "<protocol-version>" UNSIGNED "</protocol-version>" ;

  OPERATING-SYSTEM ::= "<operating-system>" TEXT "</operating-system>" ;

  PLATFORM ::= "<platform>" TEXT "</platform>" ;

  USER-AGENT ::= "<user-agent>" TEXT "</user-agent>" ;

  HTML-FRAME-PATH ::= "<html-frame-path>" TEXT "</html-frame-path>" ;

  STOP-TYPE ::= "<stop-type>"
                  ( "script" | "exception" | "error" | "abort" )
                "</stop-type>" ;

  SCRIPT-TYPE ::= "<script-type>"
                     ( "inline" | "event" | "linked" | "timeout" | "java" | "generated" | "unknown" )
                  "</script-type>" ;

  THREAD-TYPE ::= "<thread-type>"
                     ( "inline" | "event" | "linked" | "timeout" | "java" | "unknown" )
                  "</thread-type>" ;

  NAMESPACE ::= "<namespace>" TEXT "</namespace>" ;

  # The event type is e.g., "click", "mousemove"
  # More examples are at http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html
  # Exactly which events are implemented depends on the host, and is not defined in this protocol.

  EVENT-TYPE ::= "<event-type>" TEXT "</event-type>" ;

  STATUS ::= "<status>"
               ( "completed" | "unhandled-exception" | "aborted" | "cancelled-by-scheduler" )
             "</status>" ;

  DATA-TYPE ::= "<data-type>"
                 ( "number" | "boolean" | "string" | "null" | "undefined" | "object" )
               "</data-type>" ;

  MODE ::= "<mode>"
             ( "run" | "step-into-call" | "step-next-line" | "step-out-of-call" )
           "</mode>" ;

  # "broken" is sent in response to a BREAK command.
  # "breakpoint" is sent when the script hits a debugger-set breakpoint.

  STOPPED-REASON ::= "<stopped-reason>"
                       ( "broken" | "function-return" | "exception" | "debugger statement" | "breakpoint" | "unknown" )
                     "</stopped-reason>" ;

  LINE-NUMBER ::= "<line-number>" UNSIGNED "</line-number>" ;

  ARGUMENT-OBJECT ::= "<argument-object>" UNSIGNED "</argument-object>" ;

  VARIABLE-OBJECT ::= "<variable-object>" UNSIGNED "</variable-object>" ;

  THIS-OBJECT ::= "<this-object>" UNSIGNED "</this-object>" ;

  ###
  # Generic context-free data:
  #
  # A TAG represents a value passed from the client to the host and
  # returned from the host with a reply.
  #

  TAG ::= "<tag>" UNSIGNED "</tag>" ;

  YESNO ::= "<yes/>" | "<no/>" ;

  STRING ::= "<string>" TEXT "</string>" ;

  URI ::= "<uri>" TEXT "</uri>" ;

  ###
  # Identifiers:  Most of these are globally unique (they're just the
  # integer representation of some pointer).  THREAD-ID and PARENT-THREAD-ID
  # are relative to a RUNTIME-ID.  FRAME-ID is relative to the current stack
  # height in a stopped thread: 0 being the top-most frame (i.e., the most
  # recently called), 1 being the caller for that, and so on.
  #

  RUNTIME-ID ::= "<runtime-id>" UNSIGNED "</runtime-id>" ;

  OBJECT-ID ::= "<object-id>" UNSIGNED "</object-id>" ;

  # The window ID is shared across scope. Notably, it's the same as in the console logger and window manager
  # INTERNAL: The value is from Window::id

  WINDOW-ID ::= "<window-id>" UNSIGNED "</window-id>" ;

  PROTOTYPE-ID ::= "<prototype-id>" UNSIGNED "</prototype-id>" ;

  FUNCTION-ID ::= "<function-id>" UNSIGNED "</function-id>" ;

  SCRIPT-ID ::= "<script-id>" UNSIGNED "</script-id>" ;

  BREAKPOINT-ID ::= "<breakpoint-id>" UNSIGNED "</breakpoint-id>" ;

  HANDLER-ID ::= "<handler-id>" UNSIGNED "</handler-id>" ;

  FRAME-ID ::= "<frame-id>" UNSIGNED "</frame-id>" ;

  THREAD-ID ::= "<thread-id>" UNSIGNED "</thread-id>" ;

  PARENT-THREAD-ID ::= "<parent-thread-id>" UNSIGNED "</parent-thread-id>" ;

  ###
  # Primitive data:
  #
  # You may *NOT* assume that an UNSIGNED received from the host fits
  # in 32 bits, but you may assume that 64 bits is enough.
  #
  # You must *NOT* send an UNSIGNED to the host that does not fit in 32
  # bits unless it was received from the host.
  #

  UNSIGNED ::= [0-9]+ ;

  TEXT ::= BASE64-ENCODED-DATA | textual-data ;

  BASE64-ENCODED-DATA ::= "<base64-encoded-data>" textual-data "</base64-encoded-data>" ;

Critique

The USER-AGENT value is not actually static but can be different with every request. It is strictly speaking an attribute of the script, not of the debugging host. In practice we cannot know the user agent used for all scripts, but for "inline" and "linked" we do.

The tagging system is probably too weak to support multiple clients: tags will need to contain information about the specific client that sent the command. There are other ways to fix this, e.g., by making one (designated) client send a message on behalf of another, but fixing the tagging would probably be better.

(The tagging failure may be a generic weakness in the way the protocols work, but most services won't have multiple clients so it is not so visible elsewhere.)