Module documentation for scope

$Id$

Introduction

The scope module provides an infrastructure for building a multitude of debugging tools, some for internal use at Opera and some to be shipped with Opera as "developer tools".

The most famous user of scope is dragonfly, but other tools are also available from kaleidoscope. At present these are opwatir (Opera's implementation of watir) and pyscope (a python client for scope mainly used by scope developers).

The tools can live partly inside Opera, partly outside -- and the exact division depends mostly on the needs of the tool. Read the requirements and the architecture for more information.

Tutorials

How to use the tool suite
Tutorial on how to compile Opera for debugging, how to compile the proxy and the tools, and how to run them -- enough to get you started.
How to add a new service
Tutorial on how to add a new service to Opera and some superficial generalities about how to write a client.

Requirements, design, protocols, and reference material

Overall documentation

Requirements for the debugging infrastructure
Primary and secondary requirements for the tool suite.
Architecture overview
Overview of the "client/server with proxy" architecture that is used for the tools.
XML protocols style guide
Some hints about what XML-based protocols should look like.
Security issues
Discusses security issues in remote debugging and how they may be overcome.
HTTPD interface to the proxy
Discusses how to access the proxy through its HTTP service using JavaScript and XmlHttpRequest.

Logger tools

HTTP Header Logger
Requirements and protocol for a tool that logs and displays HTTP header traffic.
Console Logger
Requirements and protocol for a tool that receives all text written to the Opera Console.
ECMAScript Logger
Requirements and protocol for a tool that receives all source code compiled by Opera's ecmascript engine.

Interactive/controlling tools

Window manager and filtering
This service enables the client to see which windows are open and set filters on which windows it wants information from. The filter will work on other services as well.
ECMAScript Debugger
Requirements, protocol description, and how-to for an interactive tool that allows script execution to be monitored and altered.
CSS Inspector
Requirements and protocol for a tool that allows monitoring and altering of CSS properties on DOM elements.
Probedata server
Requirements, protocol description, and how-to for an interactive tool that allows probetools data to be obtained on-line.
Opera Exec Conductor
Protocol description and simple examples for an interactive or automated tool to make Opera do various things that can be useful to e.g. QA testing.
URL Player
Requirements and protocol description for a tool to load URLs into Opera remotely, allowing for simple URL playing on any device.

Other reference material

Java client library
Overview of the Java utility library that's available to clients.
Javascript interface for scope
Reference for the javascript API to scope, which is included in desktop builds—both "proper" and core-gogi versions.
Transport protocol
Definition of the on-the-wire protocol used for communication between Opera, the debugging clients, and the proxy. Useful mostly if you're writing your own libraries for putting data on the wire.

Documents that need to be written

(None at present.)

Tool ideas

All tools described above have at least been partially implemented at this time. This section contains some ideas for other tools.

For site developers (and internal use)

DOM inspector

Part of the projected developer tools suite.

Mostly for developers and QA; may be useful for others

Command interface

A "command interface" to Opera generalizes some of the other tools, allowing commands to be sent to Opera and results to be retrieved. Commands may be things like opening and closing windows and loading URLs into them, running selftest, uploading and downloading content (consider asking for opera:cache from a tool). This is related to what the current probetools shell infrastructure does.

Benchmark runner

A benchmark runner allows content to be uploaded, run in a controlled environment, and results reported; there may eg be JS callbacks for reporting results measured by JS, and ways of obtaining statistics gathered during the run.

There can be benchmarks built into Opera, like selftests are, and they can perhaps be run through this infrastructure.

Selftest runner

Runs some selection of the built-in selftests. Useful for automated regression testing, probably.

Cache monitor (Tahera / Sigbjørn)

This tool sends logging data about cache activity: what goes into the cache, what is removed from the cache, what is replaced. Information would be at least: URL, age, expiry information, cache-related HTTP headers plus a timestamp.

Memory monitor

This tool interacts with probetools / memtools to extract and display memory profiling information, on-line and off-line. Most likely this will partly be a port of the memory displays from happy-malloc, the core-gogi memory logging, and the Synergy "Gem" infrastructure.

General resource logging framework

This generalizes other tools, allowing parts of Opera easily to register with the framework when they want to log resource "movement". See the cache monitor for one example; another use case is platform independent logging of open sockets / files, and platform dependent logging of eg graphics resources.

We would control this as we control the DBG functionality, through a config file that enables specific resource names to be tracked, and the interface could be DBG-like macros.

Pettern references adjunct\m2\src\backend\p2p\bt-util.h, and suggests resource type, address, source file and line number as reasonable parameters (or to be extracted automatically).

Plugin monitor

This tool logs all calls into and out of plugins running inside Opera, and perhaps also allows breakpoints to be set on these calls, so that they may be debugged interactively.

TCP/IP logger (Sigbjørn)

We'd be restricted to inspecing Opera's TCP connections (and support code might have to be written for each socket implementation). Utility: unknown, certainly nonzero.

Page contents tracker (Sigbjørn)

Have some way of extracting from Opera every item linked from a document: images, scripts, css, plugin content, etc.

Page extractor (Sigbjørn)

External interface to Opera to extract an entire page with all the context needed to load it in some other context without accessing the net. Probably impossible in practice, but one can probably get close.

Ties in as much with the "director" idea as anything -- mostly the problem here is to implement the functionality in Opera, not to write the tool itself.

Director

This is a tool that runs on the same machine as the host and is responsible for starting Opera and detecting when it crashes. Parameters can be passed to the director and used when Opera is started, esp things like memory size and window size.

(No doubt QA has tools like this already.)

For developers only

"DEBUG" logger

Captures the output from Opera's internal debug infrastructure, normally enabled only in debug builds. Probably quite useful for developers on embedded systems.

Message profiler and logger

Captures messages in Opera's message queue, keeps running profile of how much time is spent per message (mean/median/range), displays message queue when asked.

Applications

kibitz (eddy)

The basic idea of kibitz is to let clients be able to see what a user is doing in a tab; simplistically, a tab is created in the client's instance of Opera which mimics the behaviour of a tab in the host's instance of opera, except that it's view-only - trying to interact with it directly has no effect, though you can move it around if it's not maximised, or put other tabs in front of it; zoom in/out would scale the whole tab up/down. Ideally, the host user gets to pick which tabs are thus exposed, but it would probably suffice to expose the active tab - with slightly weird effects for the clients when the user switches tabs.

Use cases:

Note: There has been talk about implementing the same application using Alien (a.k.a. webserver). That might be a better platform, but in any case: Make sure no one else is implementing this concurrently!