Module org.htmlunit
Package org.htmlunit

Class WebRequest

java.lang.Object
org.htmlunit.WebRequest
All Implemented Interfaces:
Serializable
public class WebRequest extends Object implements Serializable
Parameter object for making web requests.
Author:
Brad Clarke, Hans Donner, Ahmed Ashour, Marc Guillemot, Rodney Gitzel, Ronald Brill, Adam Afeltowicz, Joerg Werner, Michael Lueck, Lai Quang Duong, Kristof Neirynck
See Also:

  • Constructor Details

    • WebRequest

      public WebRequest(URL url, String acceptHeader, String acceptEncodingHeader)
      Instantiates a WebRequest for the specified URL, setting the Accept and Accept-Encoding headers if provided.
      Parameters:
      url - the target URL
      acceptHeader - the accept header to use
      acceptEncodingHeader - the accept encoding header to use

    • WebRequest

      public WebRequest(URL url, Charset charset, URL refererUrl)
      Instantiates a WebRequest for the specified URL, using the given charset and referer.
      Parameters:
      url - the target URL
      charset - the charset to use
      refererUrl - the url be used by the referer header

    • WebRequest

      public WebRequest(URL url)
      Instantiates a WebRequest for the specified URL.
      Parameters:
      url - the target URL

    • WebRequest

      public WebRequest(URL url, HttpMethod submitMethod)
      Instantiates a WebRequest for the specified URL using the given HTTP submit method.
      Parameters:
      url - the target URL
      submitMethod - the HTTP submit method to use

  • Method Details

    • newAboutBlankRequest

      public static WebRequest newAboutBlankRequest()
      Returns a new request for about:blank.
      Returns:
      a new request for about:blank

    • getUrl

      public URL getUrl()
      Returns the target URL.
      Returns:
      the target URL

    • setUrl

      public void setUrl(URL url)
      Sets the target URL. The URL may be simplified if needed (for instance eliminating irrelevant path portions like "/./").
      Parameters:
      url - the target URL

    • getProxyHost

      public String getProxyHost()
      Returns the proxy host to use.
      Returns:
      the proxy host to use

    • setProxyHost

      public void setProxyHost(String proxyHost)
      Sets the proxy host to use.
      Parameters:
      proxyHost - the proxy host to use

    • getProxyPort

      public int getProxyPort()
      Returns the proxy port to use.
      Returns:
      the proxy port to use

    • setProxyPort

      public void setProxyPort(int proxyPort)
      Sets the proxy port to use.
      Parameters:
      proxyPort - the proxy port to use

    • getProxyScheme

      public String getProxyScheme()
      Returns the proxy scheme to use.
      Returns:
      the proxy scheme to use

    • setProxyScheme

      public void setProxyScheme(String proxyScheme)
      Sets the proxy scheme to use.
      Parameters:
      proxyScheme - the proxy scheme to use

    • isSocksProxy

      public boolean isSocksProxy()
      Returns whether SOCKS proxy or not. Returns whether SOCKS proxy or not.
      Returns:
      whether SOCKS proxy or not

    • setSocksProxy

      public void setSocksProxy(boolean isSocksProxy)
      Sets whether SOCKS proxy or not.
      Parameters:
      isSocksProxy - whether SOCKS proxy or not

    • getTimeout

      public int getTimeout()
      Returns the timeout to use.
      Returns:
      the timeout to use

    • setTimeout

      public void setTimeout(int timeout)
      Sets the timeout to use.
      Parameters:
      timeout - the timeout to use

    • getEncodingType

      public FormEncodingType getEncodingType()
      Returns the form encoding type to use. Returns the form encoding type to use.
      Returns:
      the form encoding type to use

    • setEncodingType

      public void setEncodingType(FormEncodingType encodingType)
      Sets the form encoding type to use.
      Parameters:
      encodingType - the form encoding type to use

    • getParameters

      public List<NameValuePair> getParameters()
      Retrieves the request parameters used. Similar to the servlet api function getParameterMap() this works depending on the request type and collects the url parameters and the body stuff.
      The value is also normalized - null is converted to an empty string.

      In contrast to the servlet api this creates a separate KeyValuePair for every parameter. This means that pairs with the same name can be part of the list. The servlet api will return a string[] as value for the key in this case.
      Additionally this method includes also the uploaded files for multipart post requests.

      Returns:
      the request parameters to use

    • getRequestParameters

      public List<NameValuePair> getRequestParameters()
      INTERNAL API - SUBJECT TO CHANGE AT ANY TIME - USE AT YOUR OWN RISK.
      Returns the request parameters to use. If set, these request parameters will overwrite any request parameters which may be present in the URL. Should not be used in combination with the request body.
      Returns:
      the request parameters to use

    • setRequestParameters

      public void setRequestParameters(List<NameValuePair> requestParameters) throws RuntimeException
      INTERNAL API - SUBJECT TO CHANGE AT ANY TIME - USE AT YOUR OWN RISK.
      Sets the request parameters to use. If set, these request parameters will overwrite any request parameters which may be present in the URL. Should not be used in combination with the request body.
      Parameters:
      requestParameters - the request parameters to use
      Throws:
      RuntimeException - if the request body has already been set

    • getRequestBody

      public String getRequestBody()
      Returns the body content to be submitted if this is a POST request. Ignored for all other request types. Should not be used in combination with request parameters.
      Returns:
      the body content, or null if not set

    • setRequestBody

      public void setRequestBody(String requestBody) throws IllegalStateException
      Sets the body content to be submitted if this is a POST, PUT or PATCH request. Other request types result in IllegalStateException. Should not be used in combination with request parameters.
      Parameters:
      requestBody - the body content to be submitted if this is a POST, PUT or PATCH request
      Throws:
      IllegalStateException - if the request parameters have already been set or this is not a POST, PUT or PATCH request.

    • getHttpMethod

      public HttpMethod getHttpMethod()
      Returns the HTTP submit method to use.
      Returns:
      the HTTP submit method to use

    • setHttpMethod

      public void setHttpMethod(HttpMethod submitMethod)
      Sets the HTTP submit method to use.
      Parameters:
      submitMethod - the HTTP submit method to use

    • getAdditionalHeaders

      public Map<String,String> getAdditionalHeaders()
      Returns the additional HTTP headers to use.
      Returns:
      the additional HTTP headers to use

    • setAdditionalHeaders

      public void setAdditionalHeaders(Map<String,String> additionalHeaders)
      Sets the additional HTTP headers to use.
      Parameters:
      additionalHeaders - the additional HTTP headers to use

    • isAdditionalHeader

      public boolean isAdditionalHeader(String name)
      Returns whether the specified header name is already included in the additional HTTP headers. The comparison is case-insensitive.
      Parameters:
      name - the header name to look up
      Returns:
      true if the header is present; false otherwise

    • getAdditionalHeader

      public String getAdditionalHeader(String name)
      Returns the value of the specified additional HTTP header, or null if the header is not set. The name comparison is case-insensitive.
      Parameters:
      name - the header name to look up
      Returns:
      the header value, or null if not present

    • setRefererHeader

      public void setRefererHeader(URL url)
      Sets the Referer HTTP header to the external form of the given URL. Does nothing if the URL is null or does not use the http or https scheme.
      Parameters:
      url - the URL to use as the referer, or null

    • getFetchDestination

      public WebRequest.FetchDestination getFetchDestination()
      Returns the destination of this request, used to compute the Sec-Fetch-Dest header (and, unless overridden, the default Sec-Fetch-Mode). Defaults to WebRequest.FetchDestination.EMPTY, which is correct for plain XMLHttpRequest/fetch() calls.
      Returns:
      the destination of this request

    • setFetchDestination

      public void setFetchDestination(WebRequest.FetchDestination fetchDestination)
      Sets the destination of this request.
      Parameters:
      fetchDestination - the destination of this request, or null to reset to WebRequest.FetchDestination.EMPTY

    • getFetchModeOverride

      public WebRequest.FetchMode getFetchModeOverride()
      Returns the explicit mode override for this request, if any. When null (the default), the mode is derived from the destination.
      Returns:
      the mode override, or null if none was set

    • setFetchModeOverride

      public void setFetchModeOverride(WebRequest.FetchMode fetchMode)
      Sets an explicit mode override for this request, e.g. for a fetch() call using an explicit mode option, or a subresource request using the crossorigin attribute (which forces CORS mode).
      Parameters:
      fetchMode - the mode to use, or null to derive it from the destination

    • isUserActivation

      public boolean isUserActivation()
      Returns whether this request is the result of a navigation backed by genuine user activation (e.g. a click on a link, a typed URL, or a form submitted via a click on its submit button) as opposed to one triggered purely by script (e.g. location.href = ..., a <meta http-equiv="refresh">, or an automatically-loaded <iframe>).

      Only relevant for requests whose Sec-Fetch-Mode is navigate; used to compute the presence of the Sec-Fetch-User header, which real browsers omit entirely (never send as ?0) whenever this is false.

      Returns:
      whether this request was triggered by a real user gesture

    • setUserActivation

      public void setUserActivation(boolean userActivation)
      Sets whether this request is the result of a navigation backed by genuine user activation.
      Parameters:
      userActivation - whether this request was triggered by a real user gesture

    • getRequestingUrl

      public URL getRequestingUrl()
      Returns the URL of the document or script that initiated this request, used to compute the Sec-Fetch-Site header. null means there is no initiator (e.g. a browser-chrome-initiated navigation such as a typed URL or bookmark), which maps to Sec-Fetch-Site: none.

      Note this is tracked separately from the Referer header: unlike the referrer, it must not be affected by referrer-policy stripping, since Sec-Fetch-Site always reflects the true relationship between the initiator and the target, even when no Referer header is sent.

      Returns:
      the URL of the initiator, or null if there is none

    • setRequestingUrl

      public void setRequestingUrl(URL requestingUrl)
      Sets the URL of the document or script that initiated this request.
      Parameters:
      requestingUrl - the URL of the initiator, or null if there is none

    • markAsNavigation

      public void markAsNavigation(URL requestingUrl, boolean userActivation)
      Convenience method for the common case of a top-level navigation (an anchor/area click, a form submission, a script-driven location change, ...): sets WebRequest.FetchDestination.DOCUMENT, the initiator URL, and whether the navigation was backed by genuine user activation, all in one call.

      Every navigation-triggering call site needs all three of these set together for correct Sec-Fetch-* headers; bundling them here makes it harder for a call site to set some of them and forget the rest.

      For navigations whose destination is not WebRequest.FetchDestination.DOCUMENT (e.g. an <iframe>/<frame> load), use markAsNavigation(FetchDestination, URL, boolean) instead.

      Parameters:
      requestingUrl - the URL of the page initiating this navigation, or null if there is none (e.g. a typed URL)
      userActivation - whether this navigation was triggered by a real user gesture as opposed to script

    • markAsNavigation

      public void markAsNavigation(WebRequest.FetchDestination destination, URL requestingUrl, boolean userActivation)
      Same as markAsNavigation(URL, boolean), but for navigations whose destination isn't a top-level WebRequest.FetchDestination.DOCUMENT - currently only <iframe>/<frame> loads (WebRequest.FetchDestination.IFRAME/ WebRequest.FetchDestination.FRAME).
      Parameters:
      destination - the navigation's destination
      requestingUrl - the URL of the page initiating this navigation, or null if there is none
      userActivation - whether this navigation was triggered by a real user gesture as opposed to script

    • setAdditionalHeader

      public void setAdditionalHeader(String name, String value)
      Sets the specified name/value pair in the additional HTTP headers, replacing any existing header with the same name (case-insensitive).
      Parameters:
      name - the header name
      value - the header value

    • removeAdditionalHeader

      public void removeAdditionalHeader(String name)
      Removes the specified header from the additional HTTP headers. The name comparison is case-insensitive. Does nothing if the header is not present.
      Parameters:
      name - the header name to remove

    • getUrlCredentials

      public org.apache.http.auth.Credentials getUrlCredentials()
      Returns the credentials extracted from the URL's userinfo component (e.g. http://user:secret@host/), or null if none were present.
      Returns:
      the URL-embedded credentials, or null

    • getCredentials

      public org.apache.http.auth.Credentials getCredentials()
      Returns the credentials explicitly set via setCredentials(Credentials), or null if none were set.
      Returns:
      the explicitly configured credentials, or null

    • setCredentials

      public void setCredentials(org.apache.http.auth.Credentials credentials)
      Sets the credentials to use.
      Parameters:
      credentials - the credentials to use

    • getCharset

      public Charset getCharset()
      Returns the character set to use to perform the request.
      Returns:
      the character set to use to perform the request

    • setCharset

      public void setCharset(Charset charset)
      Sets the character set to use to perform the request. The default value is StandardCharsets.ISO_8859_1.
      Parameters:
      charset - the character set to use to perform the request

    • getDefaultResponseContentCharset

      public Charset getDefaultResponseContentCharset()
      Returns the default character set to use for the response when it does not specify one.
      Returns:
      the default character set to use for the response when it does not specify one.

    • setDefaultResponseContentCharset

      public void setDefaultResponseContentCharset(Charset defaultResponseContentCharset)
      Sets the default character set to use when the response does not declare one explicitly.

      Unless overridden, the default is StandardCharsets.ISO_8859_1 (per HTTP/1.1).

      Parameters:
      defaultResponseContentCharset - the fallback charset; must not be null

    • hasHint

      public boolean hasHint(WebRequest.HttpHint hint)
      Returns whether the given WebRequest.HttpHint is currently enabled for this request.
      Parameters:
      hint - the hint to check
      Returns:
      true if the hint is enabled; false otherwise

    • addHint

      public void addHint(WebRequest.HttpHint hint)
      Enables the given WebRequest.HttpHint for this request.
      Parameters:
      hint - the hint to enable

    • toString

      public String toString()
      Overrides:
      toString in class Object