Web Caching

March 2003

Web Caching 1

Objectives
When you have completed this module
you will be able to do the following:
Configure NetCache for web caching
Configure NetCache for FTP
Configure NetCache for DNS caching

March 2003

Web Caching 2

Web Caching Overview

HTTP caching
behavior

FTP configuration

HTTP General
configurations
Cacheability Control
by ACLs
IMS (If Modified
Since) rules
Header rules
HTTP Tunneling

FTP over HTTP

FTP caching behavior

DNS
Cache Content pre-fill

March 2003

Web Caching 3

Setup > HTTP > General

Setup > HTTP > General

This page contains the options to specify general enabling and authentication settings for
HTTP services on this NetCache appliance.
Refer to Online Help for specific configuration information.

March 2003

Web Caching 4

HTTP General contd

Client Connection Timeout

Used to specify a period of inactivity after which NetCache drops a client connection.
Using this option will retire resources on a NetCache and allow active connections by
others.

HTTP Proxy Ports

Used to specify port numbers on which the NetCache appliance listens for client Web
cache requests. You can enter multiple ports in this option.

URLs to Rewrite
Transparently redirects requests for URLs to local mirror sites that you specify. To use
this option, specify the URL for which you want to redirect requests and the URL to
which the requests should be redirected.

Refer to Online Help for specific configuration information.

March 2003

Web Caching 5

HTTP General contd

Client Privacy Enable

If you enable this option, NetCache does not send the client's IP address (in the XForwarded-For MIME header) when attempting a fetch from a Web server. If you disable
this option, NetCache sends client IP addresses to the server.

Do Not Report Errors to Selected Clients

Select this option to suppress NetCache error messages referring to proxy-cache server
presence that are sent to the requesting client.

Suppress Writing Via Header

If this option is enabled, reference to this appliance is not appended to the Via header for
HTTP requests serviced through this appliance or for HTTP objects served by this
appliance.

Client- Side Persistence Enable

If this option is enabled, NetCache will maintain a client connection beyond the servicing
of a single request.

March 2003

Web Caching 6

Server- Side Persistence Enable

If this option is enabled, NetCache will maintain a Web server connection beyond the
servicing of a single request

March 2003

Web Caching 7

Fine Grained HTTP

Cacheability Control
ACLs used to refine caching behavior
ACL variables allowed:
server-ip, server-name, server-port, clientip, client-port, date, time, url, url-prefix

Use ACL as the underlying engine, but

interface is separate

March 2003

Web Caching 8

HTTP Caching Behavior

HTTP Caching Behavior

Options on the Setup > HTTP > Caching Behavior page enable you to control how
cacheable and uncacheable objects are handled, to increase cache performance by
defining how URLs with cookies are handled, and to redirect URL requests to local
mirror sites.
Note: Options on this page are not applicable to a NetCache appliance configured only as
a news cache.
Refer to Online Help for specific configuration information.

March 2003

Web Caching 9

Caching Behavior contd

Ignore URL Trailing Characters Substring List

Example
If you enter the following URL substring for this option:
http://www.aaa.com/
NetCache will cache objects with the following URLs and trailing strings as the same
object under the URL,
http://www.aaa.com/asdfadf:
http://www.aaa.com/asdfadf?dddd
http://www.aaa.com/asdfadf?ffff

March 2003

Web Caching 10

Caching Behavior contd

TTL Patterns
^http:// 1440
^http:// 0 10% 1300

HTTP objects (strings that begin with http://) are cached for 1440
minutes.
HTTP objects are cached for 10% of the object's age or a
maximum of 1,300 minutes, whichever is less.

March 2003

Web Caching 11

Caching Behavior contd

Strip Cookies from Objects Cached by URL

If this option is enabled, NetCache strips the set-cookie header out of any cached object
containing a cookie so that the cookie does not get passed to clients requesting that
object.

Treating Requests with no Cookie

If a client requests a currently-cached object containing a cookie but does not include a
cookie in its request, this option specifies how NetCache handles that request.

March 2003

Web Caching 12

Cacheable Cookie URLs

Specifies source URLs whose fetched objects will be cached on this NetCache appliance
even if they contain cookies.

Examples
http://www.xyz.com cookiexzyx

Caches all objects that are fetched from

http://xyz.com and contain the cookie, cookies
and the cookie value, zyx.

xyz.com * *

Caches all cookie containing objects that are

fetched from URLs with the substring xyz.com.

* cookiex *

Caches all objects that contain the cookie,

cookiex.

Cookie Cheat File Name Extensions

Specifies that all objects that end in the extensions listed are cacheable, even with a
cookie.

March 2003

Web Caching 13

IMS Rules

IMS
If-Modified-Since. A query that NetCache sends to a Web server in order to verify the
freshness of an object, that is, to determine whether the object has been modified.

IMS Rules tab options

The Caching Behavior IMS Rules tab page displays the IMS (If-Modified-Since) rules
that determine the maximum time (in seconds) between object refreshes in this NetCache
appliance. No object can exceed this time between refreshes. Maximum age limits from
the origin server that specify longer times are overridden by this setting.

March 2003

Web Caching 14

GUI: IMS rules contd

Default rule cannot be disabled or deleted

Rule number can be changed to reflect
precedence

March 2003

Web Caching 15

IMS Rules contd

Use Wizard Mode

Select this option to specify an explicit caching rule using the GUI options described
below. NetCache will use the selections you make to generate a CLI formatted explicit
caching rule. If you prefer to write an explicit caching rule directly, unselect this option
and select the "UseAdvanced Mode" option.

Use Advance Mode

Select the advanced mode option if you prefer to write this explicit caching rule directly.
Then enter the explicit caching rule directly in the text box. Writing the explicit caching
rule directly enables you to specify more than one factor to determinethe set of cached
objects to which the rule applies.
Syntax:
{on|off} {cache|no-cache|uncond-cache} [if] [date="<mo> <date>
<year>"]
[and] [time= <hr>:<min>{am|pm}-<hr>:<min>{am|pm}] [and]
[{server-ip|server-name|server-port|client-ip|clientport|url|url-prefix|content-type} {=|!=|contains|matches}
<value>]

March 2003

Web Caching 16

GUI: Explicit Caching Rules

Explicit Caching Rules tab

Displays rules that determine which objects are cached.

March 2003

Web Caching 17

GUI: Explicit Caching contd

Advanced mode is useful for multiple conditional phrases

Rules edited in wizard mode will be displayed in both modes
Advanced mode rules can only be displayed/edited in advanced
mode

March 2003

Web Caching 18

GUI: Explicit Caching contd

March 2003

Web Caching 19

Response Header
Setup > HTTP > Header and Caching Behavior Header Rules

Header Rules
Header Rules tab page displays rules that determine objects from which to delete and/or
insert response headers based on specified actions and conditions.

March 2003

Web Caching 20

Setup > HTTP > Tunneling

HTTP - Tunneling
This feature is used to specify general enabling and authentication settings for HTTP
tunneling services on the NetCache appliance.
Refer to Online Help for specific configuration information.

March 2003

Web Caching 21

Setup > FTP > General

FTP Modes
Use options on the Setup tab > FTP > General page to configure this NetCache
appliance for either FTP service over HTTP or transparent FTP service. To complete FTP
service configuration, you must also configure options located elsewhere in the NetCache
Manager Setup tab, as follows:

Nontransparent FTP enable FTP Proxy

If you are setting up FTP over HTTP, you must also configure NetCache for HTTP
service (Setup tab > HTTP pages).

If you are setting up transparent FTP, you must also configure the FTP transparency
options (Setup tab > Network > Transparency page).

March 2003

Web Caching 22

FTP over HTTP

Setup > FTP > General

Options Specific to FTP over HTTP

For FTP over HTTP, NetCache masquerades as an FTP client. FTP clients must provide a
user name and password to the FTP server. When a NetCache appliance is deployed, the
NetCache transfers commands on behalf of the client and receives the responses from the
server. To do this, Netcache must be authenticated with the server.
Refer to Online Help for specific configuration information.

March 2003

Web Caching 23

Setup > FTP > Caching Behavior

FTP Caching Behavior

This page contains options for managing objects for FTP over HTTP and transparent
FTP.
Refer to Online Help for specific configuration information.

March 2003

Web Caching 24

Setup >DNS > General

DNS Caching
When DNS caching is enabled, this NetCache appliance acts as a DNS cache for clientinitiated DNS lookups. NetCache caches DNS A- type records (a mapping of host name
to IP addresses) and proxies all other DNS-related information. After NetCache caches a
host name and IP address, NetCache can resolve a DNS query for the same host.

March 2003

Web Caching 25

HTTP Content Fill

confill [options] <srcurl> [dsturl]

Example
confill http://netcache.com/
confill http://netcache.com/ http://remote.com/

Lock Age, Max Age and Chunked Encoding Not

Implemented

March 2003

Web Caching 26

Web Caching Exercises

Change HTTP proxy ports on NetCache and
client browser
Web Caching Behavior Configurations
Configure HTTP Fine Grained Cacheablity
Build an Explicit Caching Rule
Configure IMS Rules

March 2003

Web Caching 27

Web Caching Exercises

15 minutes in length
Utilizing Breakout rooms
Instructor will visit all rooms
Broadcast announcement 5 minutes
prior to regroup
Stay focussed, start telnet, start GUI
Share microphones, or no one else can
be heard

March 2003

Web Caching 28

Web Caching Exercise

Objectives
When you have completed this module you will be able to do the following:

Change http proxy ports on NetCache and client browser

Locate Web Caching behavior configurations

Configure Complete Fetch

Set Default Maximum Age and Force Refresh Rate

Configure Cookie Cheat extensions

Configure and test HTTP fine-grained cacheablity.

Exercise Overview
The purpose of this activity is for you to perform the procedures introducing you to
NetCache web caching configuration and behavior. During these exercises, you will be
able to go through each step in the process, and will have an opportunity to verify that
each step was successfully completed.

Time Estimate: 30 minutes

Required Hardware, Software, and Tools
Hardware

Workstation

NetCache appliance

Software

NetCache 5.4 or later

Netscape Navigator 4.7

Internet Explorer 5.5 or later

March 2003

Web Caching 29

Change HTTP proxy ports on NetCache and client browser

1.

Access the NetCache browser interface (http://netcacheX:3132).

2.

Select Setup > HTTP > General.

3.

Scroll down to HTTP Proxy Ports and observe that the default port is 3128.

4.

On the next line, type 80 so that 3128 and 80 are in a column

5.

Commit Changes.

6.

On your browser, Internet Explorer or Netscape, access the proxy settings.

See instructions in Admin Basics Exercise if necessary.

7.

Change the proxy port on your browsers proxy setting from 3128 to 80.
Note:

If the browser proxy port number and the NetCache HTTP Proxy port
numbers do not agree the browser will not be able to access the
NetCache

Web Caching Behavior Configurations

This is a getting acquainted exercise. Observe the Configure Complete Fetch,
Set Default Maximum Age, Force Refresh Rate, and Configure Cookie Cheat extensions
Configurations. Observe the configurations, note default values, and check the help
screens.
1.

Select Setup > HTTP > Caching Behavior

2.

Observe Caching Behavior configurations:

Complete Fetch not set

TTL Pattern Enable

TTL Patterns

Cookie Cheat File Name Extensions: jpg, jpeg, gif

Note: Other configurations, such as Accept-Ranges header forwarding,
Ignore no-cache directive, and 4.1 Compatible Caching Enable on the
Web Caching Behavior Page are highly specialized to specific
environments. Some of these configurations should be changed from
the default setting only with careful consideration and perhaps
advisement from Network Appliance support engineers.

March 2003

Web Caching 30

HTTP Fine Grained Cacheablity

TTL patterns and TTL override:
To enable this capability:
1.

At the NetCache command line, set

config.http.caching.ttl.enable on (on by default)

2.

. What does this do?

Confirm that the Web Server content is set to expire in 60 seconds.

3.

Request content from the server and examine the cached object to verify the
TTL = 60 seconds

Examine the Cached object

4.

Open a browser client and ensure it is proxied through your NetCache.

5.

Request an object form the Web server by entering:

http://pdc/cache/secondtest.htm

6.

Then confirm that the TTL set for the object is shown to be one minute by
accessing the NetCache command line and entering:
netcache>object http://pdc/cache/secondtest.htm

7.

Verify that the following is displayed:

March 2003

Web Caching 31

Properties:
URL: http://pdc/cache/secondtest.htm
LastVerify: Wed, 23 Jan 2002 11:31:20 PST
NextVerify: Wed, 23 Jan 2002 11:32:20 PST
MinAge: <not set>

MaxAge: 60 seconds (0d 0h 1m 0s)

MaxLockAge: <not set>
Response headers:
HTTP/1.0 200 OK
Age: 34
Accept-Ranges: bytes
Date: Wed, 23 Jan 2002 19:31:20 GMT
Content-Length: 323
Content-Type: text/html
Expires: Wed, 23 Jan 2002 19:32:20 GMT
Cache-Control: max-age=60
Connection: keep-alive
Server: Microsoft-IIS/5.0
Last-Modified: Wed, 23 Jan 2002 19:24:08 GMT
ETag: "b014608743a4c11:a18"
Via: 1.1 C1100-ZCH01840381 (NetCache
NetApp//n/dremel2/rlse7/netcache/nightl
y/RcvN_020120_0830)

Exhibit the Modified Behavior

The Web server has been configured to set the TTL to one minute.
8.

Create the new TTL pattern

netcache>set
on"

config.http.caching.ttl.patterns "^http:// 120

This sets the time to live to 2 hours or 7200 seconds.

9.

Eject the object

netcache>eject http://pdc/cache/secondtest.htm

March 2003

Web Caching 32

Ejected http:// 10.32.70.10/cache/secondtest.htm

10.

Verify that the object is no longer in the NetCache.

netcache>object http://pdc/cache/secondtest.htm
http://pdc/cache/secondtest.htm' is not in the cache.

11.
12.

Request the object again by doing a forced reload of the same

Use the object command and show the new TTL.

NetCache>object http://pdc/cache/secondtest.htm
NetCache>object http://pdc/cache/secondtest.htm
Properties:
URL: http://10.32.70.10/cache/secondtest.htm
LastVerify: Wed, 23 Jan 2002 13:05:40 PST
NextVerify: Wed, 23 Jan 2002 15:05:40 PST
MinAge: <not set>
MaxAge: 7200 seconds (0d 2h 0m 0s)
MaxLockAge: <not set>
Response headers:
HTTP/1.0 200 OK
Age: 29
Accept-Ranges: bytes
Date: Wed, 23 Jan 2002 21:05:40 GMT
Content-Length: 323
Content-Type: text/html
Expires: Wed, 23 Jan 2002 23:05:40 GMT
Cache-Control: max-age=7200
Connection: keep-alive
Server: Microsoft-IIS/5.0
Last-Modified: Wed, 23 Jan 2002 19:24:08 GMT
ETag: "b014608743a4c11:a18"
Via: 1.1 C1100-ZCH01840381 (NetCache
NetApp//n/dremel2/rlse7/netcache/nightl

y/RcvN_020120_0830)

March 2003

Web Caching 33

12.

Observe that there is an entry in the NetCache Web Access log similar to the
following:

1011819952.22 0.001 - TCP_HIT_EJECT/- - PUSH

http://10.32.70.10/cache/secondtest.htm admin - "" - 1011819956.33
0.030 10.41.18.4 TCP_MISS_PRIVATE_STATUS_CODE/304 318 GET
http://10.32.70.10/cache/secondtest.htm - DIRECT/10.32.70.10 "" 1011819966.65 0.001 10.41.18.4 TCP_MISS/200 732 GET
http://10.32.70.10/cache/secondtest.htm - DIRECT/10.32.70.10
"text/html" - 1011820159.47 0.001 10.41.18.4 TCP_HIT_IMS_NOTMOD/304 194
GET http://10.32.70.10/cache/secondtest.htm - - "text/html"

Stop here! Notify instructor.

Inform the instructor that you have completed the first part of this exercise. Once
everyone has reached this point, the instructor will modify the HTTP header as required
for the next exercise.

March 2003

Web Caching 34

Build an Explicit Caching Rule

1.

2.

Build an explicit caching rule to prove that we can unconditionally cache the
result of a request for content from a site that normally does not get cached:
Prior to testing, please confirm that the instructor has introduced a "CacheControl: no-cache" custom HTTP Header.
At the NetCache prompt, enter:
show config.http.caching.cacheability_rule_list
What was the result?

3.

Use your proxied browser client to go to:

http://pdc/cache/thirdtest.htm

4.
5.

View the Web Access log and notice that this object was a miss.
Force the browser to reload the URL and view the results in the Web Access
Log.
What was this result?

6.

Go to the NetCache Manager (or the command line) and create a rule to
unconditionally cache:
http://pdc/cache/thirdtest.htm

7.
8.
9.

Make this new rule the first rule in the list.

Return to the proxied browser client and force it to fetch the URL from the
server.
Go to the NetCache command line and show the new object.
netcache>object http://10.32.70.10

March 2003

Web Caching 35

(64.94.95.10)/cache/thirdtest.htm
10.
11.

Request the object again, then verify that it delivered from the NetCache.
Use either the CLI object command or the Web Access Log and verify that
the object of interest is in NetCache.

March 2003

Web Caching 36

IMS Rules
Build a rule that says no IMS
1.

Write an IMS rule that will ignore any IMS requirements for objects served
from the IIS server. You may use the NetCache Manager
(Setup>HTTP>Caching Behavior> IMS Rules) or the following command
line entries:
netcache>show config.http.caching.ims_rule_list
config.http.caching.ims_rule_list = \\
on ims every 31536000 and obey-httphdr
on no-ims if server-ip = 10.32.70.10
\\

2.
3.

Modify the default cacheablity rule shown above to establish IMS every 60
seconds.
Open a browser, proxy thru NetCache, then go to URL:
http://pdc/cache/fourthtest.htm

4.

Look at Web Access Log and view Last 45 lines, and observe no IMS steps,
as the second time access will just be a hit.

5.

Go to two or three external Web sites (other than the pdc) to generate some
log traffic.

6.
7.

Wait at least one minute (the IMS period).

Reload from the browser and look at the Web Access View Last 45 Lines.
We are looking for:
TCP_MISS_RELOAD/200 448 GET
http://10.32.70.10/cache/fourthtest.htm - DIRECT/10.32.70.10
"text/html" - 1011633561.33 0.001 10.41.18.4

TCP_HIT_IMS_NOTMOD/304 204 GET

http://10.32.70.10/cache/fourthtest.htm - DIRECT/10.32.70.10
"text/html" -

March 2003

Web Caching 37

What does the 304 and 204 mean? Describe it here.

For a description of the HTTP 1.1 error codes see:

http://www.w3.org/Protocols/rfc2068/rfc2068.txt

March 2003

Web Caching 38

March 2003

Web Caching 39