<?xml version="1.0" standalone="no"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
 "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">
<book>
  <bookinfo>
    <title>WebScarab</title>
  </bookinfo>
  <chapter id="about">
    <title>About WebScarab</title>
    <para>WebScarab is a development project, run under the
      auspices of the Open Web Application Security Project
      (OWASP). The aim of WebScarab is to provide a free (beer as
      well as speech) tool for web application developers and
      reviewers to use to understand the functioning of web
      applications, and to identify possible problems that could
      cause those web applications to malfunction.
    </para>
    <para>WebScarab is licensed under the GNU General Public
      License v2.
    </para>
    <para>WebScarab is based on a plugin architecture. Currently,
      the plugins are mostly security focused, but I'm sure that
      "normal" web developers would also be able to benefit from
      some of the plugins, such as the Proxy and Manual Request
      plugins.
    </para>
    <para>WebScarab is a powerful tool, and as with many tools, can
      be used for good as well as for bad. In the hands of the
      white-hats, WebScarab can help them to find
      vulnerabilities, and make sure that those vulnerabilities
      are fixed as soon as possible. In the hands of the
      black-hats, WebScarab allows them to find and possibly
      exploit vulnerabilities.
    </para>
    <para>
      <emphasis>Disclaimer</emphasis>
    </para>
    <para>WebScarab is a tool, nothing more. It was created with
      the best intentions. The author(s) cannot be held
      responsible for what other people choose to use it for.
    </para>
  </chapter>
  <chapter id="introduction">
    <title>Introduction</title>
    <para>WebScarab is basically a framework for running plugins.
      It provides a foundation for the plugins to generate
      requests  and responses (aka conversations), keep an audit
      record of
      these conversations, and review what those conversations
      were,  as well as a tree view of the URL's that have been
      "seen".
    </para>
    <para>It also passes all of the conversations that the
      framework sees to all of the plugins that are installed.
      This  allows each plugin to perform some analysis of each
      conversation.
    </para>
    <para>Other services that the framework itself provides are
      handling of upstream proxy servers, client side
      certificates, a pool of cookies that have been seen or set
      by various
      plugins,  and a convenient way to perform various encodings
      and
      decoding  of strings.
    </para>
  </chapter>
  <chapter id="sessions">
    <title>WebScarab session management</title>
    <para>WebScarab uses the concept of a session to relate various
      conversations. It is compulsory to have a session when
      using WebScarab, because each conversation is written to
      the session  as it is seen, and read back from the session
      when
      requested.
    </para>
    <para>At the moment, sessions can only be saved to a directory
      on the filesystem. When creating a new session, you should
      select an empty directory, or provide the name of a
      directory  that should be created.
    </para>
    <para>When WebScarab starts up, it prompts you to choose what
      kind of session to use. The default option is to create a 
      temporary session. This temporary session is a directory
      created  in the ${java.io.tmpdir} directory, using a
      pattern of 
      webscarabnnnnn.tmp.
    </para>
    <para>
      <emphasis>Note:</emphasis>This temporary directory is
      deleted  when WebScarab exits. If you wish to preserve the
      audit
      record  for later review, you should start a new session
      before
      creating or intercepting any conversations. Alternatively,
      if  you need to save a temporary session that already
      contains
      conversations, you should <emphasis>NOT</emphasis> exit
      WebScarab.  First change to a  new session, or open an
      existing session. This causes
      WebScarab  to flush some memory-based data structures to
      disk. Then
      copy  or move the temporary session directory to a new
      location.  
    </para>
    <para>
      <emphasis>Note:</emphasis>It could be really useful for
      WebScarab to store its sessions in a SQL database. Each of
      the  plugins have been implemented with an interface to its
      "datastore". As mentioned, there is currently a 
      FileSystemStore
      implementation of these interfaces, which could be used as
      a template for implementing a SQLStore. For standalone
      use,  WebScarab could use HSQLDB, or could be configured to
      use
      an external database such as Oracle or PostgreSQL. This
      would be a  great feature for an interested person to
      contribute to
      WebScarab.
    </para>
  </chapter>
  <chapter id="environment">
    <title>WebScarab environmental settings</title>
    <section id="proxies">
      <title>Upstream Proxy</title>
      <para>If you need to use an upstream proxy setting, you can
        configure it under the Tools -&gt; Proxies menu.
      </para>
      <para>For Windows users, WebScarab has support for querying
        Internet Explorer's proxy settings, and automatically
        loading  them into the appropriate dialog boxes. This
        feature
        requires  a JNI plugin/DLL, which has to be on the
        PATH. If the
        DLL was  successfully located, you will see a button
        entitled
        "Get IE  Settings". If the DLL is not found, this
        button is not
        shown.  If you used the WebScarab installer to install
        WebScarab,  this should all be configured. In most
        cases, simply
        including the W32WinInet.dll file in the same directory
        as  the WebScarab JAR file should be sufficient. If you
        are
        running WebScarab via a shortcut, make sure that the
        application directory is configured to be the directory
        in  which the W32WinInet.dll and WebScarab JAR file is
        located.
      </para>
      <figure>
        <title>Upstream proxy dialog</title>
        <mediaobject>
          <imageobject>
            <imagedata fileref="proxies.png" format="PNG"/>
          </imageobject>
          <textobject>
            <phrase>Configuring upstream proxies</phrase>
          </textobject>
        </mediaobject>
      </figure>
    </section>
    <section id="clientcerts">
      <title>Client-side Certificates</title>
      <para>WebScarab has support for client side SSL
        certificates. It can access keys and certificates
        stored in PKCS#12 format files, as well as using PKCS#11 tokens
        on the Java 1.5 platform. 
        This functionality is accessed via the Tools-&gt;Certificates menu.
        Simply choose the "Add Keystore" button, and select the type of keystore to
        add. If it is a PKCS#12 store, provide the file location, and the keystore
        password. If it is a PKCS#11 keystore, you need to give the store an arbitrary name,
        as well as specifying the location of the PKCS#11 library that Java should use to access the token.
        You also need to provide the password for accessing the token.
      </para>
      <para>Once keystores have been configured, you can see the details of the certificates
        in the keystores. Clicking on the certificate alias will display the details of the certificate.
      </para>
      <para>To activate a particular certificate, select "Activate selected Alias". 
        You may be prompted for a password to use to access the key. 
        To stop using any client certificate, make sure that no alias is selected (Ctrl-Click if necessary),
        and select "Activate selected Alias".
      </para>
      <para>You can see which alias is currently active in the "Active key" text field.
      </para>
      <para><emphasis>Note:</emphasis> The fingerprint of the client certificate will be included as
        a fake header in the request when the conversation is archived. This is intended to allow
        for automatic replay of a request using the client certificate that was selected at the time the request was issued.
        If you wish to replay the request with a different (or no) certificate, make sure to 
        delete the &quot;X-SSLClientCertificate&quot; header from the request.
      </para>
    </section>
    <section id="authentication">
      <title>server and Proxy authentication</title>
      <para>WebScarab has support for authenticating against proxy and web servers, using
        Basic, NTLM and Negotiate (not Kerberos) authentication. The first time a server requests 
        authentication, WebScarab will prompt for credentials. These credentials will be reused for 
        subsequent requests when required by the web server or proxy.
      </para>
      <para><emphasis>Note:</emphasis> The credentials used will be included as
        a fake header in the request when the conversation is archived. This is intended to allow
        for automatic replay of a request using the credentials that were active at the time the request was issued.
        If you wish to replay the request with different (or no) credentials, make sure to 
        delete the &quot;Authorization&quot; or &quot;Proxy-Authorization&quot; header from the request.
      </para>
    </section>
  </chapter>
  <chapter id="plugins">
    <title>WebScarab Plugins</title>
    <section>
      <title>WebScarab plugins</title>
      <para>WebScarab uses the concept of plugins, which can do
        one  or both of the following: 
        <itemizedlist>
          <listitem>Create conversations</listitem>
          <listitem>Analyse conversations</listitem>
        </itemizedlist>
      </para>
      <para>Plugins that create conversations use a 
        plugin-specific
        method to decide which resources to request from the 
        server,
        how to parameterise the request headers, etc, then 
        submits
        the request to the server. It then receives the 
        response from
        the server, can perform some computation on the 
        response, and
        can then decide whether or not to submit that 
        conversation to
        the framework.  
      </para>
      <para>All conversations that are submitted to the framework
        are distributed to all of the plugins that are 
        installed in
        the framework. Each plugin then gets the opportunity to
        perform some analysis on that conversation.
      </para>
      <para>Current plugins include:</para>
      <para>
        <itemizedlist>
          <listitem>
            <link linkend="proxy">Proxy</link>
          </listitem>
          <listitem>
            <link linkend="manualrequest">Manual Request</link>
          </listitem>
          <listitem>
            <link linkend="webservices">Web Services</link>
          </listitem>
          <listitem>
            <link linkend="spider">Spider</link>
          </listitem>
          <listitem>
            <link linkend="sessionid">SessionID Analysis</link>
          </listitem>
          <listitem>
            <link linkend="scripted">Scripted</link>
          </listitem>
          <listitem>
            <link linkend="fragments">Fragments</link>
          </listitem>
          <listitem>
            <link linkend="compare">Compare</link>
          </listitem>
          <listitem>
            <link linkend="fuzzer">Fuzzer</link>
          </listitem>
          <listitem>
            <link linkend="search">Search</link>
          </listitem>
        </itemizedlist>
      </para>
    </section>
    <section id="proxy">
      <title>The Proxy plugin</title>
      <para>The proxy plugin sets up a HTTP proxy, listening by
        default on localhost port 8008. In order to use this
        listener, you should configure your browser to use
        WebScarab
        as an upstream proxy. Once this is done, any requests
        that
        your browser makes will be routed through webScarab,
        and will
        be recorded and analysed.
      </para>
      <para>WebScarab also has support for intercepting
        SSL-encrypted HTTPS requests. If your browser is
        configured
        to use WebScarab for SSL requests, WebScarab will use
        its
        own, untrusted SSL server certificate to negotiate an
        encrypted session with the browser, in order to read
        the request that the browser is making to the
        "secure" server.
        Naturally, since the certificate that WebScarab uses is
        not
        trusted by your browser, your browser SHOULD warn you
        that
        the certificate is untrusted, and give you the option
        to continue.
      </para>
      <para>
        <emphasis>Note:</emphasis>This ability to observe what
        your
        browser is sending over an SSL connection does not mean
        that
        "SSL has been broken". Your browser should have given
        you the
        necessary warnings. If you choose to ignore the
        warnings, it
        is not a failure of SSL.
      </para>
      <section id="listeners">
        <title>Proxy listeners</title>
        <para>WebScarab supports multiple HTTP listeners. The
          Listener allows you to specify the IP address and
          port that
          the Listener listens on. 
          <emphasis>Note:</emphasis>By default, WebScarab
          listens
          only on localhost, so as to eliminate the
          possibility of
          unauthorized anonymous proxying.
        </para>
        <para>WebScarab allows you to specify a "base address"
          for
          a Listener. The base address instructs the Listener
          to
          operate as a reverse proxy, and should be formatted
          as a
          HTTP or HTTPS URL. In this mode, it will act as a
          web
          server, rather than as a proxy server, and will
          construct
          the URL by concatenating the base URL and the path
          that
          appears in the request line. If the base URL is an
          HTTPS
          URL, it will immediately negotiate an SSL tunnel
          prior to
          trying to read the request from the browser. This
          is useful
          for the situation where you are using a custom
          HTTP-based
          client that does not support configuring an
          upstream proxy. 
          Simply change the hosts file on the computer on
          which the 
          custom client is running to point the site in
          question to 
          the computer on which WebScarab is running on, and
          WebScarab 
          will receive requests for the targeted website. <emphasis>
            Note:
          </emphasis>
          You can't run both WebScarab and the client
          application
          on the same computer in such a situation, because
          WebScarab will 
          get the same IP address from the hosts file when
          trying to connect
          to the upstream target, and will end up looping
          back to itself 
          recursively. <emphasis>Note:</emphasis>You SHOULD
          also have to add the
          WebScarab certificate to the client applications
          list of recognised
          certificates, to prevent certificate errors from
          occuring. If you
          DON'T have to do this, the client application is
          vulnerable to a 
          man in the middle attack, because it does not
          verify the certificate
          properly.
        </para>
        <para>The network simulator allows the user to limit
          the bandwidth available to the client, and
          introduces
          artificial latencies.
        </para>
        <para id="enableplugins">There is also an option to
          enable plugins. 
          If this option is NOT selected, the various proxy
          plugins will NOT be
          invoked when each request and response is seen. If
          the plugins
          do not seem to be working, or your intercepts are
          not happening,
          check to make sure that this checkbox is selected.
        </para>
        <para>On the Windows platform, if the W32WinInet DLL is
          loaded by WebScarab, the Listener that is marked as
          primary
          will automatically "hijack" the Internet Explorer
          when it
          starts, and configure Internet Explorer (and all
          other
          WinInet based applications) to use WebScarab as a
          proxy.
          When the Listener stops (for example, if it is
          manually
          stopped, or if WebScarab exits), the original
          settings will
          be restored. Obviously, it makes no sense to
          configure
          multiple Primary listeners, as they will conflict
          with each
          other, and probably result in the wrong proxy
          settings
          being restored when WebScarab is closed. <emphasis>
            Note:
          </emphasis>
          At this point, there is nothing technically
          preventing you 
          from configuring multiple "Primary" listeners. Just
          don't do it.
          Also, it makes no sense to configure a Listener
          with a base URL,
          as well as setting it as a primary listener,
          because it will 
          be talking a different protocol (expecting
          non-fully-qualified
          URL's, instead of URL's that include the scheme and
          host portion).
          Just don't do it.
        </para>
      </section>
      <section>
        <title>Active conversations</title>
        <para>Below the Listener configuration is a table which
          shows recent and current requests which have been
          handled
          by the various Listeners. This can be useful when
          troubleshooting connectivity errors, since the
          proxy will
          not save failed requests to the general Summary.
        </para>
        <para>A useful option might be to have a specific
          option for increasing the debugging output
          in the Proxy plugin,
          for detailed debugging
        </para>
      </section>
      <section id="proxyplugins">
        <title>Proxy plugins</title>
        <para>It is possible to write proxy plugins that
          perform various modifications to the
          requests and responses in
          transit through the proxy. Current plugins are: 
          <itemizedlist>
            <listitem>
              <link linkend="proxyintercept">Manual
                Intercept
              </link>
            </listitem>
            <listitem>
              <link linkend="proxybeanshell">Bean Shell</link>
            </listitem>
            <listitem>
              <link linkend="revealhidden">Reveal hidden
                form
                fields
              </link>
            </listitem>
            <listitem>
              <link linkend="preventcaching">Prevent
                browser caching
                content
              </link>
            </listitem>
            <listitem>
              <link linkend="injectcookies">Inject known
                cookies into
                requests
              </link>
            </listitem>
            <listitem>
              <link linkend="collectcookies">Extract
                cookies from
                responses
              </link>
            </listitem>
            <listitem>
              <link linkend="ntlmfilter">Remove NTLM
                authentication
                headers
              </link>
            </listitem>
          </itemizedlist>
        </para>
        <section id="proxyintercept">
          <title>Manual Intercept</title>
          <para>The Manual Intercept proxy plugin allows the
            user
            to intercept requests from the browser and
            responses from
            the server, inspect them, and optionally modify
            them
            before transmission. This is particularly
            useful when one
            wants to sumbmit a form to a webserver, but
            JavaScript
            validation rejects the values that you would
            like to
            submit.The request can be altered after the
            validation
            has been performed.
          </para>
          <para>The user has the option to intercept requests
            with
            specific methods, by selecting from the
            "Methods" list.
            If one wants to select more than a single
            method, use
            Ctrl when clicking to select additional items.
          </para>
          <para>The user also has the ability to specify a
            Java
            Regular Expression that the URL must match in
            order to be
            intercepted. You also have the option to
            specify a
            regular expression that matches URLs that
            should not be
            intercepted. Once can also specify whether the
            Regular
            Expression should be evaluated in a Case
            Sensitive
            fashion, for instance where the image file
            extension is
            .GIF
          </para>
          <para>WebScarab can also intercept responses coming
            from
            the server, and modify those before they are
            relayed to
            the browser. You can specify a Regular
            Expression that
            the response Content-Type header should match
            in order to
            be intercepted.
          </para>
        </section>
        <section id="proxybeanshell">
          <title>Bean Shell</title>
          <para>The Bean Shell proxy plugin enables scripted
            modifications of the request and response. This
            is
            particularly useful when one wants to perform
            the same
            modification to a number of conversations, or
            if the
            modification is particularly complex. The
            script has
            complete access to the request before it is
            sent to the
            server, and the response before it is sent to
            the
            browser. The request and response objects are
            currently
            only fully documented in the JavaDoc for the
            relevant
            classes (org.owasp.webscarab.model.Request and
            org.owasp.webscarab.model.Response).
          </para>
          <para>
            <emphasis>Note :</emphasis>Changes made to the
            script in
            the editor window only take effect when the
            "Commit"
            button is pressed.
          </para>
        </section>
        <section id="proxymisc">
          <title>Miscellaneous plugins</title>
          <para>There are a number of miscellaneous plugins
            that
            can operate on proxied conversations. 
            <itemizedlist>
              <listitem id="revealhidden">The "Reveal
                hidden fields" plugin changes any
                forms fields with the type of "hidden"
                to have a type
                of "text". This causes the browser to
                display these
                fields, and makes it easy for the user
                to modify the
                fields, as well as observing when such
                hidden fields
                are being used.
              </listitem>
              <listitem id="preventcaching">The "prevent
                caching" plugin removes any
                "if-modified-since" or similar headers
                from requests to
                ensure that WebScarab always has a copy
                of the response
                body, rather than allowing the browser
                to use a locally
                cached copy.
              </listitem>
              <listitem id="injectcookies">The "Inject
                cookies" plugin allows WebScarab
                to override any cookies in the browser,
                or to supply
                cookies that the browser does not know
                about. The
                cookies are retrieved from the "Shared
                cookies" list,
                and are filtered for applicability
                according to the
                domain and path attributes.
              </listitem>
              <listitem id="collectcookies">The "Collect
                cookies" plugin extracts any
                "Set-Cookie" headers from responses
                passing through the
                proxy, and adds those to the "Shared
                cookies" list.
                these can later be used by other
                plugins to maintain
                state, if desired.
              </listitem>
              <listitem id="ntlmfilter">NTLM
                authentication cannot be used through a
                proxy. However, since browsers should
                use the strongest
                authentication method available,
                Internet Explorer will
                usually try to use NTLM authentication
                in favour of
                Basic, and will fail. However, IE will
                not then fall
                back to using Basic authentication.
                WebScarab works
                around this problem by removing or
                modifying any
                Authenticate headers that offer the
                NTLM protocol. If
                your upstream proxy or target web
                server require NTLM
                authentication you may want to try
                using APS (the
                authenticating proxy server) which DOES
                support NTLM
                authentication. It may be possible to
                run both
                WebScarab and APS in a chained
                configuration (with APS
                closer to the target). This has not
                been tested,
                however. 
                <emphasis>Note:</emphasis>
                Interestingly enough, it <emphasis>
                  IS
                </emphasis>
                possible to perform SPNEGO
                authentication through a
                proxy. In particular, since WebScarab
                supports
                persistent connections, it is possible
                to monitor
                proxied conversations to a server that
                uses SPNEGO,
                although each portion of the protocol
                negotiation
                between the browser and server will be
                recorded, and
                will result in a number of 401
                responses ending up in
                the logs. This should even work for
                intercepted SSL
                connections!
              </listitem>
            </itemizedlist>
          </para>
        </section>
      </section>
    </section>
    <section id="manualrequest">
      <title>The Manual Request plugin</title>
      <para>The Manual Request plugin allows the user to
        handcraft a reuest to be sent to the server. It is
        also possible to
        replay a previous request by selecting it from the drop
        down
        selection box. Previous requests which are loaded into
        the
        editor can also be edited before being sent to the
        server, of
        course. When the "Fetch Response" button is selected,
        WebScarab sends the request to the appropriate server,
        and
        saves the conversation for analysis by the other
        WebScarab
        plugins.
      </para>
      <para>Selecting "Get cookies" button gets cookies relevant
        to
        the requested URL from the "Shared Cookies" list, and
        adds
        them to the request.
      </para>
      <para>The "Update CookieJar" button looks for any
        "Set-Cookie" headers in the response that was
        retrieved, and
        adds them to the "Shared Cookies" list.
      </para>
    </section>
    <section id="webservices">
      <title>The Web Services plugin</title>
      <para>The Web Services plugin is intended to allow the user 
      to manually consume a SOAP web service. The plugin automatically 
      identifies conversations that contain WSDL documents, and
      presents them in a drop down list for selection.</para>
      <para>One can also type in the URL manually, which will 
      result in the plugin retrieving the WSDL. If the WSDL is 
      not accessible via an HTTP or HTTPS url, the operator can also 
      access a local copy of the WSDL by simply providing the path to the
      local copy in the WSDL URL field, and pressing Load.</para>
      <para>The plugin then presents the available services and 
      operations, and automatically populates a tree containing
      the parameters that the service requires.</para>
      <para>The operator can then press the &quot;Execute&quot; button
      to submit the request the server, and receive the response.</para>
      <para>At the moment, the web services plugin is still very immature,
      and is likely to be buggy. At the very least, this initial implementation
      only has support for RPC/encoded transports.</para>
    </section>
    <section id="spider">
      <title>The Spider plugin</title>
      <para>The Spider plugin analyses responses to identify any
        links in the response body, or the "Location" header.
        If the
        URL represented has not been seen, the URL is added to
        a tree, and can be automatically downloaded when
        desired.
        WebScarab has two modes of fetching unseen links.
        "Fetch Tree" enumerates all currently unknown
        links below the
        selected node, and queues them for retrieval. "Fetch
        Selection" queues only the selected nodes for
        retrieval.
      </para>
      <para>Future enhancements should also identify HTML forms,
        and
        present those to the user for manual modification and
        submission.
        This would be a great project for an interested user to
        implement.
      </para>
    </section>
    <section id="sessionid">
      <title>Session ID Analysis</title>
      <para>Session ID Analysis is useful to determine how easy
        it is for
        an attacker to brute force a victim's sessionid, and
        take it over.
      </para>
      <para>The principle is that we collect a sample of session
        identifiers,
        most likely from a cookie that is set in the browser,
        but possibly also one that
        is set in a hidden form field in the HTML body.
        WebScarab tags all sessionid's 
        with the date and time that it was collected, and then,
        after performing some
        calculations on the string value to convert it into a
        number, plots the
        value against time on a graph. The human eye is a lot
        more efficient at identifying
        patterns than a computer, so by plotting the values on
        a pair of axes, we 
        make it easy for a human to visualise the sequence.
      </para>
      <para>The default calculator identifies the possible
        character set 
        for each position in the session id string. It then
        performs base-n conversion
        of the string to get it to a large number. This is
        quite an efficient way
        of converting arbitrary strings into a number, as it
        does not make any
        assumptions about the character set, and is reasonably
        effective at eliminating
        constant strings (they are reduced to a value of 0
        since the character 
        set does not change)
      </para>
      <para>It is possible to use alternative calculators, e.g. a
        Base64 calculator
        that knows what the Base64 character set is, but there
        is no User Interface support
        for selecting different calculators. It currently
        requires code changes to use
        different calculators. This might be a good project for
        an 
        interested person to implement.
      </para>
      <section id="sessionidcollection">
        <title>Collecting sessionids</title>
        <para>There are currently two ways to collect
          sessionids, from a cookie in the
          response headers, and by matching a regular
          expression against the response body.
        </para>
        <para>To collect sessionids from cookies, make sure
          that the "From message body"
          checkbox is NOT selected. To collect sessionids
          from the message body, make sure
          that the "From message body" checkbox IS selected.
          Provide a name for the series of
          sessionids (the name is calculated automatically
          for cookies), and a regular 
          expression that can be used to identify the
          identifier from the rest of the message.
        </para>
        <para>The regular expression MUST match the ENTIRE
          message body, so it should probably
          start and end with ".*" constructs. The portion of
          the matched text that should be used
          for the session identifier should be surrounded by
          round brackets (i.e. a group). 
          WebScarab will concatenate multiple groups into a
          single identifier if desired.
        </para>
        <para>Example regular expressions:
          <itemizedlist>
            <listitem>.*id="(..........)".* will extract a
              10-character sessionid 
              surrounded by quotes, and prefixed by "id="
            </listitem>
          </itemizedlist>
        </para>
        <para>Once you have specified what you want to collect,
          you need to provide
          a request that will result in a response that
          contains a new sessionid. The
          easiest way of doing this is to look in the
          SummaryPanel to see conversations
          that have a value in the "Set-Cookie" column. Then,
          select that request from the
          drop-down menu, and use that exact request to
          collect new sessionids.
        </para>
        <para>Alternatively, you can hand-craft your request,
          possibly changing a GET to
          HEAD to reduce the amount of data that needs to be
          requested.
        </para>
        <para>Once the request has been entered, you can test
          the setup to make sure
          that it works. Press the "Test" button. If a
          sessionid is successfully found,
          a dialog box will be shown with the details. Once
          the sessionids are being 
          successfully extracted via the "Test" button, you
          can specify a number of sessionids
          to collect in the "Samples" field, and click
          "Fetch" to start the process
        </para>
        <para>If you specify a large number of sessionids, and
          want to stop the collection
          process before it finishes naturally, enter "0"
          into the "Samples" field, and press "Fetch".
        </para>
      </section>
      <section id="sessionidanalysis">
        <title>Session ID Analysis</title>
        <para>While the sessionids are being collected, you can
          change to the
          Analysis tab, and view the collected values. Using
          the "Session Identifier"
          drop-down box, choose the identifier that you are
          interested in. The table
          below will fill with the identifiers that have been
          collected, and will extend
          as new identifiers are seen. The first column
          specifies the date/time that the 
          sessionid was collected, then the actual string
          value of the identifier is 
          in the second column. The third column shows the
          calculated value, which may change
          over time, as the "per-position character-set" is
          updated. The final
          column shows the difference between subsequent
          calculated values.
        </para>
        <para>If you want to copy the sessionids into a
          spreadsheet program for alternative
          analysis, you can simply select, Copy (Ctrl-C) and
          then paste it into your program.
        </para>
      </section>
      <section id="sessionidvisualisation">
        <title>Session ID Visualisation</title>
        <para>The "Visualisation" tab contains a graph view of
          the selected session 
          identifier. The graph widget is created the first
          time the tab is selected.
          If no session identifier is selected on the
          "Analysis" tab, no points will be
          shown on the graph. Using the graph, it should be
          easy to identify linear or
          repeating patterns in the session ids collected.
        </para>
      </section>
    </section>
    <section id="scripted">
      <title>The scripted plugin</title>
      <para>This plugin is intended to give users the ability to
        create test scripts,
        using the facilities of Webscarab.
      </para>
      <para>The scripting support is provided by the
        Bean Scripting Framework (BSF), which means that any
        BSF-supported scripting
        language can be used to write test scripts. The default
        language supported
        is BeanShell, which is a relaxed Java-like language.
        For more details of BeanShell,
        go to http://www.beanshell.org/. If you wish to use one
        of the other scripting
        languages such as Jython or Groovy, for example, you
        should place the appropriate
        support libraries in the ClassPath prior to running
        WebScarab, and make sure that
        your script's filenames end with the appropriate
        extension for your chosen
        language. The required extension for BeanShell scripts
        is ".bsh".
      </para>
      <para>The Scripted plugin exports a Java Object with
        methods that allow you to
        interact with the rest of WebScarab, for example,
        fetching a response, or
        adding a conversation to the WebScarab framework for
        analysis by the other plugins.
        It is likely that this object will change, as it is a
        pretty new feature of WebScarab.
        Therefore, if the documented behaviour does not match
        what you observe, please check
        the source code!
      </para>
      <para>
        The exported Object is documented in
        org/owasp/webscarab/plugins/scripted/ScriptedObjectModel.java
        The major methods that you will likely want to use are:
        <itemizedlist>
          <listitem>
            Request <function>getRequest(int id)</function>
          </listitem>
          <listitem>
            Response <function>fetchResponse(Request
              request) throws IOException
            </function>
          </listitem>
          <listitem>
            boolean <function>hasAsyncCapacity()</function>
          </listitem>
          <listitem>
            void <function>submitAsyncRequest(Request
              request)
            </function>
          </listitem>
          <listitem>
            boolean <function>isAsyncBusy()</function>
          </listitem>
          <listitem>
            boolean <function>hasAsyncResponse()</function>
          </listitem>
          <listitem>
            Response <function>getAsyncResponse() throws
              IOException
            </function>
          </listitem>
          <listitem>
            ConversationID <function>
              addConversation(Response response)
            </function>
          </listitem>
        </itemizedlist>
        The "async" methods allow for fetching of Responses in
        multiple threads, while the 
        "fetchResponse" method would be used for
        single-threaded operations.
      </para>
    </section>
    <section id="fragments">
      <title>The fragments plugin</title>
      <para>The fragments plugin parses HTML responses, and looks
        for scripts and comments.
        These can be interesting, first to see if there are any
        hidden links, or other
        debugging code, and also to understand how the pages
        are supposed to operate.
        The drop down list at the top allows you to select
        between the two types of 
        fragment, and the list immediately below shows unique
        fragments in that category
        that have been identified. Selecting a particular
        fragment in the list updates 
        the conversation table below, showing the various
        conversations that included
        that fragment.
      </para>
      <para>The Summary panel also indicates which conversations
        and which URL's have 
        comments and scripts in them. If you right-click on the
        conversation in question,
        you can select a menu option that will show all the
        fragments of the specified
        type in that particular conversation.
      </para>
    </section>
    <section id="compare">
      <title>The Compare plugin</title>
      <para>The Compare plugin allows the user to judge the
        degree of difference between
        a number of responses. This is useful when you have
        issued a number of requests 
        for a particular URL, possibly via the Scripted plugin,
        and would like to evaluate
        the results. Obviously, if you can eliminate groups of
        identical responses at once,
        you can save yourself some time.
      </para>
      <para>Choose the "base" conversation against which others
        should be compared, using 
        the drop down list. The Conversation table will be
        populated with conversations,
        and the Difference column will indicate the number of
        words that differ in the 
        responses. The number is actually the Levenshtein Edit
        Distance, showing the number
        of edits (inserts, changes, and deletes) that are
        required to transform
        the base response into the current target. For speed
        reasons, the response content
        is tokenised into words, since the Levenshtein
        algorithm is O(n*m), and slows
        down quickly for large numbers of comparisons.
      </para>
      <para>The two responses are then displayed side by side in
        the lower half
        of the window, to make it easy to compare them.
      </para>
    </section>
    <section id="fuzzer">
      <title>The Fuzzer plugin</title>
      <para>The fuzzer basically allows you to throw a
        combination of values at a
        server. It is still a little rough, but functionally,
        it works pretty
        well. The idea is that you configure the request
        method, the basic URL
        (without any parameters), the request version, any
        headers (e.g. a Host:
        header if you are using HTTP/1.1, etc. Don't configure
        cookies here!),
        and a list of Parameters.
      </para>
      <para>A Parameter is defined by its location (Path,
        Fragment, Query, Cookie,
        Body), its name (ignored for Path), type == String,
        Default value, Fuzz
        Priority, and a fuzz source.
      </para>
      <para>The default value is the value that will be submitted
        if no fuzz source
        is defined.
      </para>
      <para>The fuzz priority defines how the various fuzz
        sources are combined: if
        all the priorities are the same value, the number of
        queries submitted
        will be the number of items in the shortest fuzz
        source. If they are
        different values, the number of queries generated will
        be the product of
        the number of items in each level.
      </para>
      <para>An example would be if you had a list of usernames
        and passwords. If
        both username and password parameters had the same
        priority, usernames
        and passwords would be consumed in lock-step. If they
        had different
        priorities, EVERY password would be tried for each
        username.
      </para>
      <para>Parameters are processed in order, first path, then
        fragment, then
        query, then cookie, then body. The only content type
        supported for POST
        Requests is application/x-www-urlencoded, it does not
        support multi-part
        (although provision is made for parameters of different
        types, e.g.
        File), or XML documents, etc. Multi-part and XML will
        be the next evolution.
      </para>
    </section>
    <section id="search">
      <title>The Search plugin</title>
      <para>The search plugin allows you to execute arbitrary
        beanshell scripts to
        identify "interesting" conversations. You are provided
        with the request,
        response, and origin (originating plugin) of the
        conversation, and can
        use the class methods to return a true or false value.
        A true value
        indicates an interesting conversation that should be
        displayed, and a
        false value indicates that the conversation should not
        be displayed.
      </para>
      <para>
        An example might be something like:
      </para>
      <para>
        response.getContent() != null &amp;&amp; new
        String(response.getContent()).matches("(?s).*[Ee](rror|xception).*")
      </para>
      <para>
        which makes sure that the response HAS content
        (byte[]), before checking
        to see if a String constructed from that content
        contains any of the
        strings: Error, error, Exception, exception. The (?s)
        instructs the Java
        Regex algorithm to perform a multi-line match, i.e.
        enables the period
        to match a linefeed character.
      </para>
      <para>
        BeanShell also does automatic JavaBean introspection,
        so in fact, the
        call to response.getContent() could also be written as
        "response.content", with BeanShell using the getters
        and setters
        appropriately.
      </para>
      <para>
        For details of the class methods available, please
        consult the JavaDocs
        (included in the installer build) for
        org.owasp.webscarab.model.(Request|Response). Origin is
        simply a String
        matching the plugin name.
      </para>
    </section>
  </chapter>
  <chapter id="appendix">
    <title>Appendix</title>
    <section id="javadocs">
      <title>Request and Response methods</title>
      <para>Request methods that you may use are: 
        <itemizedlist>
          <listitem>String getMethod()</listitem>
          <listitem>void setMethod(String method)</listitem>
          <listitem>HttpUrl getURL()</listitem>
          <listitem>void setURL(HttpUrl url)</listitem>
          <listitem>void setURL(String url) throws
            MalformedURLException
          </listitem>
          <listitem>String getVersion()</listitem>
          <listitem>void setVersion(String version)</listitem>
        </itemizedlist>
      </para>
      <para>Response methods that you may use are: 
        <itemizedlist>
          <listitem>String getVersion()</listitem>
          <listitem>void setVersion(String version)</listitem>
          <listitem>String getStatus()</listitem>
          <listitem>void getStatus(String status)</listitem>
          <listitem>String getMessage()</listitem>
          <listitem>void setMessage(String message)</listitem>
          <listitem>String getStatusLine()</listitem>
        </itemizedlist>
      </para>
      <para>The following methods are common to both the Request
        and Response objects: 
        <itemizedlist>
          <listitem>String[] getHeaderNames()</listitem>
          <listitem>String getHeader(String name)</listitem>
          <listitem>void setHeader(String name, String
            value)
          </listitem>
          <listitem>void addHeader(String name, String
            value)
          </listitem>
          <listitem>void deleteHeader(String name)</listitem>
          <listitem>NamedValue[] getHeaders()</listitem>
          <listitem>void setheaders(NamedValue[] headers)</listitem>
          <listitem>byte[] getContent()</listitem>
          <listitem>void setContent(byte[] content)</listitem>
        </itemizedlist>
      </para>
    </section>
  </chapter>
</book>
