# NAME WWW::Mechanize - Handy web browsing in a Perl object # VERSION version 2.10 # SYNOPSIS WWW::Mechanize supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited. use WWW::Mechanize (); my $mech = WWW::Mechanize->new(); $mech->get( $url ); $mech->follow_link( n => 3 ); $mech->follow_link( text_regex => qr/download this/i ); $mech->follow_link( url => 'http://host.com/index.html' ); $mech->submit_form( form_number => 3, fields => { username => 'mungo', password => 'lost-and-alone', } ); $mech->submit_form( form_name => 'search', fields => { query => 'pot of gold', }, button => 'Search Now' ); # Enable strict form processing to catch typos and non-existant form fields. my $strict_mech = WWW::Mechanize->new( strict_forms => 1); $strict_mech->get( $url ); # This method call will die, saving you lots of time looking for the bug. $strict_mech->submit_form( form_number => 3, fields => { usernaem => 'mungo', # typo in field name password => 'lost-and-alone', extra_field => 123, # field does not exist } ); # DESCRIPTION `WWW::Mechanize`, or Mech for short, is a Perl module for stateful programmatic web browsing, used for automating interaction with websites. Features include: - All HTTP methods - High-level hyperlink and HTML form support, without having to parse HTML yourself - SSL support - Automatic cookies - Custom HTTP headers - Automatic handling of redirections - Proxies - HTTP authentication Mech is well suited for use in testing web applications. If you use one of the Test::\*, like [Test::HTML::Lint](https://metacpan.org/pod/Test%3A%3AHTML%3A%3ALint) modules, you can check the fetched content and use that as input to a test call. use Test::More; like( $mech->content(), qr/$expected/, "Got expected content" ); Each page fetch stores its URL in a history stack which you can traverse. $mech->back(); If you want finer control over your page fetching, you can use these methods. `[follow_link()](#mech-follow_link)` and `[submit_form()](#mech-submit_form)` are just high level wrappers around them. $mech->find_link( n => $number ); $mech->form_number( $number ); $mech->form_name( $name ); $mech->field( $name, $value ); $mech->set_fields( %field_values ); $mech->set_visible( @criteria ); $mech->click( $button ); [WWW::Mechanize](https://metacpan.org/pod/WWW%3A%3AMechanize) is a proper subclass of [LWP::UserAgent](https://metacpan.org/pod/LWP%3A%3AUserAgent) and you can also use any of [LWP::UserAgent](https://metacpan.org/pod/LWP%3A%3AUserAgent)'s methods. $mech->add_header($name => $value); Please note that Mech does NOT support JavaScript, you need additional software for that. Please check ["JavaScript" in WWW::Mechanize::FAQ](https://metacpan.org/pod/WWW%3A%3AMechanize%3A%3AFAQ#JavaScript) for more. # IMPORTANT LINKS - [https://github.com/libwww-perl/WWW-Mechanize/issues](https://github.com/libwww-perl/WWW-Mechanize/issues) The queue for bugs & enhancements in WWW::Mechanize. Please note that the queue at [http://rt.cpan.org](http://rt.cpan.org) is no longer maintained. - [https://metacpan.org/pod/WWW::Mechanize](https://metacpan.org/pod/WWW::Mechanize) The CPAN documentation page for Mechanize. - [https://metacpan.org/pod/distribution/WWW-Mechanize/lib/WWW/Mechanize/FAQ.pod](https://metacpan.org/pod/distribution/WWW-Mechanize/lib/WWW/Mechanize/FAQ.pod) Frequently asked questions. Make sure you read here FIRST. # CONSTRUCTOR AND STARTUP ## new() Creates and returns a new WWW::Mechanize object, hereafter referred to as the "agent". my $mech = WWW::Mechanize->new() The constructor for WWW::Mechanize overrides two of the params to the LWP::UserAgent constructor: agent => 'WWW-Mechanize/#.##' cookie_jar => {} # an empty, memory-only HTTP::Cookies object You can override these overrides by passing params to the constructor, as in: my $mech = WWW::Mechanize->new( agent => 'wonderbot 1.01' ); If you want none of the overhead of a cookie jar, or don't want your bot accepting cookies, you have to explicitly disallow it, like so: my $mech = WWW::Mechanize->new( cookie_jar => undef ); Here are the params that WWW::Mechanize recognizes. These do not include params that [LWP::UserAgent](https://metacpan.org/pod/LWP%3A%3AUserAgent) recognizes. - `autocheck => [0|1]` Checks each request made to see if it was successful. This saves you the trouble of manually checking yourself. Any errors found are errors, not warnings. The default value is ON, unless it's being subclassed, in which case it is OFF. This means that standalone [WWW::Mechanize](https://metacpan.org/pod/WWW%3A%3AMechanize) instances have autocheck turned on, which is protective for the vast majority of Mech users who don't bother checking the return value of get() and post() and can't figure why their code fails. However, if [WWW::Mechanize](https://metacpan.org/pod/WWW%3A%3AMechanize) is subclassed, such as for [Test::WWW::Mechanize](https://metacpan.org/pod/Test%3A%3AWWW%3A%3AMechanize) or [Test::WWW::Mechanize::Catalyst](https://metacpan.org/pod/Test%3A%3AWWW%3A%3AMechanize%3A%3ACatalyst), this may not be an appropriate default, so it's off. - `noproxy => [0|1]` Turn off the automatic call to the [LWP::UserAgent](https://metacpan.org/pod/LWP%3A%3AUserAgent) `env_proxy` function. This needs to be explicitly turned off if you're using [Crypt::SSLeay](https://metacpan.org/pod/Crypt%3A%3ASSLeay) to access a https site via a proxy server. Note: you still need to set your HTTPS\_PROXY environment variable as appropriate. - `onwarn => \&func` Reference to a `warn`-compatible function, such as `[Carp](https://metacpan.org/pod/Carp)::carp`, that is called when a warning needs to be shown. If this is set to `undef`, no warnings will ever be shown. However, it's probably better to use the `quiet` method to control that behavior. If this value is not passed, Mech uses `Carp::carp` if [Carp](https://metacpan.org/pod/Carp) is installed, or `CORE::warn` if not. - `onerror => \&func` Reference to a `die`-compatible function, such as `[Carp](https://metacpan.org/pod/Carp)::croak`, that is called when there's a fatal error. If this is set to `undef`, no errors will ever be shown. If this value is not passed, Mech uses `Carp::croak` if [Carp](https://metacpan.org/pod/Carp) is installed, or `CORE::die` if not. - `quiet => [0|1]` Don't complain on warnings. Setting `quiet => 1` is the same as calling `$mech->quiet(1)`. Default is off. - `stack_depth => $value` Sets the depth of the page stack that keeps track of all the downloaded pages. Default is effectively infinite stack size. If the stack is eating up your memory, then set this to a smaller number, say 5 or 10. Setting this to zero means Mech will keep no history. In addition, WWW::Mechanize also allows you to globally enable strict and verbose mode for form handling, which is done with [HTML::Form](https://metacpan.org/pod/HTML%3A%3AForm). - `strict_forms => [0|1]` Globally sets the HTML::Form strict flag which causes form submission to croak if any of the passed fields don't exist in the form, and/or a value doesn't exist in a select element. This can still be disabled in individual calls to `[submit_form()](#mech-submit_form)`. Default is off. - `verbose_forms => [0|1]` Globally sets the HTML::Form verbose flag which causes form submission to warn about any bad HTML form constructs found. This cannot be disabled later. Default is off. - `marked_sections => [0|1]` Globally sets the HTML::Parser marked sections flag which causes HTML `CDATA[[` sections to be honoured. This cannot be disabled later. Default is on. To support forms, WWW::Mechanize's constructor pushes POST on to the agent's `requests_redirectable` list (see also [LWP::UserAgent](https://metacpan.org/pod/LWP%3A%3AUserAgent).) ## $mech->agent\_alias( $alias ) Sets the user agent string to the expanded version from a table of actual user strings. `$alias` can be one of the following: - Windows IE 6 - Windows Mozilla - Mac Safari - Mac Mozilla - Linux Mozilla - Linux Konqueror then it will be replaced with a more interesting one. For instance, $mech->agent_alias( 'Windows IE 6' ); sets your User-Agent to Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) The list of valid aliases can be returned from `[known_agent_aliases()](#mech-known_agent_aliases)`. The current list is: - Windows IE 6 - Windows Mozilla - Mac Safari - Mac Mozilla - Linux Mozilla - Linux Konqueror ## $mech->known\_agent\_aliases() Returns a list of all the agent aliases that Mech knows about. This can also be called as a package or class method. @aliases = WWW::Mechanize::known_agent_aliases(); @aliases = WWW::Mechanize->known_agent_aliases(); @aliases = $mech->known_agent_aliases(); # PAGE-FETCHING METHODS ## $mech->get( $uri ) Given a URL/URI, fetches it. Returns an [HTTP::Response](https://metacpan.org/pod/HTTP%3A%3AResponse) object. `$uri` can be a well-formed URL string, a [URI](https://metacpan.org/pod/URI) object, or a [WWW::Mechanize::Link](https://metacpan.org/pod/WWW%3A%3AMechanize%3A%3ALink) object. The results are stored internally in the agent object, but you don't know that. Just use the accessors listed below. Poking at the internals is deprecated and subject to change in the future. `get()` is a well-behaved overloaded version of the method in [LWP::UserAgent](https://metacpan.org/pod/LWP%3A%3AUserAgent). This lets you do things like $mech->get( $uri, ':content_file' => $filename ); and you can rest assured that the params will get filtered down appropriately. See ["get" in LWP::UserAgent](https://metacpan.org/pod/LWP%3A%3AUserAgent#get) for more details. **NOTE:** Because `:content_file` causes the page contents to be stored in a file instead of the response object, some Mech functions that expect it to be there won't work as expected. Use with caution. Here is a non-complete list of methods that do not work as expected with `:content_file`: ` [forms()](#mech-forms) `, ` [current_form()](#mech-current_form) `, ` [links()](#mech-links) `, ` [title()](#mech-title) `, ` [content(...)](#mech-content) `, ` [text()](#mech-text) `, all [content-handling methods](#content-handling-methods), all [link methods](#link-methods), all [image methods](#image-methods), all [form methods](#form-methods), all [field methods](#field-methods), ` [save_content(...)](#mech-save_content-filename-opts) `, ` [dump_links(...)](#mech-dump_links-fh-absolute) `, ` [dump_images(...)](#mech-dump_images-fh-absolute) `, ` [dump_forms(...)](#mech-dump_forms-fh) `, ` [dump_text(...)](#mech-dump_text-fh) ` ## $mech->post( $uri, content => $content ) POSTs `$content` to `$uri`. Returns an [HTTP::Response](https://metacpan.org/pod/HTTP%3A%3AResponse) object. `$uri` can be a well-formed URI string, a [URI](https://metacpan.org/pod/URI) object, or a [WWW::Mechanize::Link](https://metacpan.org/pod/WWW%3A%3AMechanize%3A%3ALink) object. ## $mech->put( $uri, content => $content ) PUTs `$content` to `$uri`. Returns an [HTTP::Response](https://metacpan.org/pod/HTTP%3A%3AResponse) object. `$uri` can be a well-formed URI string, a [URI](https://metacpan.org/pod/URI) object, or a [WWW::Mechanize::Link](https://metacpan.org/pod/WWW%3A%3AMechanize%3A%3ALink) object. my $res = $mech->head( $uri ); my $res = $mech->head( $uri , $field_name => $value, ... ); ## $mech->head ($uri ) Performs a HEAD request to `$uri`. Returns an [HTTP::Response](https://metacpan.org/pod/HTTP%3A%3AResponse) object. `$uri` can be a well-formed URI string, a [URI](https://metacpan.org/pod/URI) object, or a [WWW::Mechanize::Link](https://metacpan.org/pod/WWW%3A%3AMechanize%3A%3ALink) object. ## $mech->reload() Acts like the reload button in a browser: repeats the current request. The history (as per the [back()](#mech-back) method) is not altered. Returns the [HTTP::Response](https://metacpan.org/pod/HTTP%3A%3AResponse) object from the reload, or `undef` if there's no current request. ## $mech->back() The equivalent of hitting the "back" button in a browser. Returns to the previous page. Won't go back past the first page. (Really, what would it do if it could?) Returns true if it could go back, or false if not. ## $mech->clear\_history() This deletes all the history entries and returns true. ## $mech->history\_count() This returns the number of items in the browser history. This number _does_ include the most recently made request. ## $mech->history($n) This returns the _n_th item in history. The 0th item is the most recent request and response, which would be acted on by methods like `[find_link()](#mech-find_link)`. The 1st item is the state you'd return to if you called `[back()](#mech-back)`. The maximum useful value for `$n` is `$mech->history_count - 1`. Requests beyond that bound will return `undef`. History items are returned as hash references, in the form: { req => $http_request, res => $http_response } # STATUS METHODS ## $mech->success() Returns a boolean telling whether the last request was successful. If there hasn't been an operation yet, returns false. This is a convenience function that wraps `$mech->res->is_success`. ## $mech->uri() Returns the current URI as a [URI](https://metacpan.org/pod/URI) object. This object stringifies to the URI itself. ## $mech->response() / $mech->res() Return the current response as an [HTTP::Response](https://metacpan.org/pod/HTTP%3A%3AResponse) object. Synonym for `$mech->response()`. ## $mech->status() Returns the HTTP status code of the response. This is a 3-digit number like 200 for OK, 404 for not found, and so on. ## $mech->ct() / $mech->content\_type() Returns the content type of the response. ## $mech->base() Returns the base URI for the current response ## $mech->forms() When called in a list context, returns a list of the forms found in the last fetched page. In a scalar context, returns a reference to an array with those forms. The forms returned are all [HTML::Form](https://metacpan.org/pod/HTML%3A%3AForm) objects. ## $mech->current\_form() Returns the current form as an [HTML::Form](https://metacpan.org/pod/HTML%3A%3AForm) object. ## $mech->links() When called in a list context, returns a list of the links found in the last fetched page. In a scalar context it returns a reference to an array with those links. Each link is a [WWW::Mechanize::Link](https://metacpan.org/pod/WWW%3A%3AMechanize%3A%3ALink) object. ## $mech->is\_html() Returns true/false on whether our content is HTML, according to the HTTP headers. ## $mech->title() Returns the contents of the `