ViSolve : Squid Configuration

LOG FILE PATH NAMES AND CACHE DIRECTORIES

Squid provides a number of logs that can be used when debugging problems, and when measuring the effectiveness and identifying users and the sites they visit. Because Squid can be used to "snoop" on users browsing habits, one should carefully consider privacy laws in your region and more importantly be considerate to your users. That's being said, logs can be very valuable tools in insuring that your users get the best service possible from your cache.

cache_dir	logformat	access_log	cache_log	cache_store_log
cache_swap_log	emulate_httpd_log	log_ip_on_direct	mime_table	log_mime_hdrs
useragent_log	referer_log	pid_filename	debug_options	log_fqdn
client_netmask

TAG NAME	cache_dir
Description	This is used to define cache directory, its path, type and size
Build Option	Default
Usage	cache_dir Type Directory-Name Mbytes Level1 Level2 [options]
Default	cache_dir ufs /usr/local/Squid/var/cache 100 16 256

Synopsis
All objects which are to be cached are stored in the disk space defined by this tag. This defines the path to cache directory, cache directory name, type and size of the cache area.

Arguments

Type

Type specifies the kind of storage system to use. Only "ufs" is built by default. To enable any of the other storage systems see the
--enable-storeio configure option.

Type is one of the following:
1. ufs is the old well-known Squid storage format that has always been there.
2. aufs uses the same storage format as ufs, utilizing POSIX-threads to avoid blocking the main Squid process on disk-I/O.This was
formerly known in Squid as async-io.
3. diskd uses the same storage format as ufs, utilizing a separate process to avoid blocking the main Squid process on disk-I/O.

Type	Usage
ufs	cache_dir ufs Directory-Name Mbytes L1 L2 [options]
aufs	cache_dir aufs Directory-Name Mbytes L1 L2 [options]s
diskd	cache_dir diskd Directory-Name Mbytes L1 L2 [options] [Q1=n] [Q2=n]

Directory-Name

Directory name is a top-level directory where cache swap files will be stored.If you want to use an entire disk for caching, then this can
be the mount-point directory. The directory must exist and be writable by the Squid process. Squid will NOT create this directory for
you.

Mbytes

Mbytes is the amount of disk space (in MB) to use under this directory. The default is 100 MB. Change this to suit your configuration

Level1

Number of first-level subdirectories which will be created under the Directory. The default is 16.

Level2

number of second-level subdirectories which will be created under each first-level directory. The default is 256.

number of unacknowledged I/O requests when Squid stops opening new files. If this many messages are in the queues, Squid won't
open new files. Default is 64.

number of unacknowledged messages when Squid starts blocking. If this many messages are in the queues, Squid blocks until it
receives some replies. Default is 72.

Option:

max-size=n

refers to the max object size this storedir supports. It is used to initially choose the storedir to dump the object.

Note: To make optimal use of the max-size limits you should order the cache_dir lines with the smallest max-size value first and the ones with no max-size specification last.

Example(s)
cache_dir ufs /cache_dir 5000 16 256

TAG NAME	logformat
Description	Defines the format for storing access logs in access.log file
Build Option	Default
Usage	logformat <name> <format specification>
Default	none

Synopsis
Using this, the default log format can be changed according to the requirement. This customizable format will be needed when you want to perform analysis on the logs stored in access.log file.

Arguments

name	Identifier holding the customized logformat
format specification	It is a string embedded with % format codes

% format codes all follow the same basic structure where all but the formatcode is optional. Output strings are automatically quoted as required according to their context and the output format modifiers are usually unneeded but can be specified if an explicit quoting format is desired. The logformat name should be added at the end of access log file in the access_log tag.

% ["|[|'|#] [-] [[0]width] [] formatcode

"	quoted string output format
[	Squid log quoted format as used by log_mime_hdrs
#	URL quoted output format
'	No automatic quoting
-	left aligned
width	field width. If starting with 0 then output is zero padded
	argument such as header name etc

Format codes:

>a	Client source IP address
>A	Client FQDN
<A	Server IP address or peer name
la	Local IP address (http_port)
lp	Local port number (http_port)
ts	Seconds since epoch
tu	subsecond time (milliseconds)
tl	Local time. Optional strftime format argument default %d/%b/%Y:%H:%M:S %z
tg	GMT time. Optional strftime format argument default %d/%b/%Y:%H:%M:S %z
tr	Response time (milliseconds)
>h	Request header. Optional header name argument on the format header[:[separator]element]
<h	Reply header. Optional header name argument as for >h
un	User name
ul	User login
ui	User ident
ue	User from external acl
Hs	HTTP status code
Ss	Squid request status (TCP_MISS etc)
Sh	Squid hierarchy status (DEFAULT_PARENT etc)
mt	MIME content type
rm	Request method (GET/POST etc)
ru	Request URL
rv	Request protocol version
et	Tag returned by external acl
ea	Log string returned by external acl
<st	Reply size including HTTP headers
<sH	Reply high offset sent
<sS	Upstream object size
%	a literal % character

Example(s)
logformat Squid %ts.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt

TAG NAME	access_log
Description	Defines the file where client activities are logged
Build Option	Default
Usage	access_log <filepath> [<logformat name> [acl acl ...]] access_log none [acl acl ...]
Default	access_log /usr/local/Squid3.0pre3/var/logs/access.log

Synopsis
All the activities the client does gets logged in this file. Using this, analysis on the access made by the clients can be done.

Customization of the logging can be done using the acl's which logs only those clauses in the specified logformat. If no acl is specified, all requests will be logged to this file.

To disable logging of a request specify "none".

Arguments

filepath	Location of the file that stores the logs
logformat name	Identifier that holds the customized log formats
acl	Filter rules to match

Example(s)
access_log /var/logs/access.log

If logformat is configured, then define the access_log as shown to get the defined logformats.

access_log /var/logs/access.log common, where common is the logformat name defined above.

TAG NAME	cache_log
Description	This tag is used to define the Cache logging file
Build Option	Default
Usage	cache_log <filepath>/filename
Default	cache_store_log /usr/local/Squid/var/logs/store.log

Synopsis

This defines the path and the file where general information about your cache's behavior goes. This file contains startup configuration information, as well as assorted error information during Squid's operation. This file is a good place to look when a website is found to have problems running through the web cache. Entries here may point towards a potential solution.

Arguments

filepath	Specifies the location of the file
filename	Actual file where the log is gathered

Example(s)
cache_log /var/logs/cache.log

TAG NAME	cache_store_log
Description	Configures the location of the caches store log file
Build Option	Default
Usage	cache_store_log <filepath>/filename
Default	cache_store_log /usr/local/Squid3.0pre3/var/logs/store.log

Synopsis

This tag defines the location where the transaction log of all objects that are stored in the object store, as well as the time when the object get deleted. This file really doesn't have very much use on a production cache, and it primarily recommended for use in debugging. Therefore, it can be turned off by entering none in the entry field.

Arguments

filepath	Specifies the location of the file
filename	Actual file where the log is gathered

Example(s)
cache_store_log /var/cache/store.log

TAG NAME	cache_swap_log
Description	Defines the filename used in each store directory to store the web caches metadata
Build Option	Default
Usage	cache_log <filepath>/filename
Default	none

Synopsis

This tag defines the file where metadata of objects saved on disk. This is a form of index for the web cache object store. These metadata is used to rebuild the cache during startup. This is not a human readable log, and it is strongly recommended to leave it in its default location on each store directory.

Note: You must give a full filename, not just a directory. Since this is the index for the whole object list you CANNOT periodically rotate it!

Arguments

filepath	Specifies the location of the file
filename	Actual file where the log is gathered

Example(s)
cache_swap_log /var/cache/cache_swap.log

TAG NAME	emulate_httpd_log
escription	Allows you to specify that Squid write its access.log in HTTPD common log file format
Build Option	Default
Usage	emulate_httpd_log on\|off
Default	emulate_httpd_log off

Synopsis

Squid write its access.log in HTTPD common log file format, such as that used by Apache and many other web servers. This allows you to parse the log and generate reports using a wider array of tools. However, this format does not provide several types of information specific to caches, and is generally less useful when tracking cache usage and solving problems. Because there are several effective tools for parsing and generating reports from the Squid standard access logs, it is usually preferable to leave this at its default of being off.

Arguments

on/off

Enable or disable this process

TAG NAME	log_ip_on_direct
Description	This tag enables/disables logging of IP adress/hostname in the access.log file
Build Option	Default
Usage	log_ip_on_direct on\|off
Default	log_ip_on_direct on

Synopsis

By making this directive to on, logs the IP Address of the destination server in the access.log file. If you want the hostname to be logged, then configure the directive to off mode.

Arguments

on/off

Enable or disable this process

TAG NAME	mime_table
Description	Used to define the file and path to the file where Squid's mime table is located
Build Option	Default
Usage	mime_table path/filename
Default	mime_table /usr/local/Squid/etc/mime.conf

Synopsis
Squid uses the mime table defined by this tag

Arguments

path	Path for the file where mime table file is located
filename	File that contains mime table

Example(s)
mime_table /usr/local//mime.conf

TAG NAME	log_mime_hdrs
Description	Enables to log extra information about clients requests in the access log
Build Option	Default
Usage	log_mime_hdrs on\|off
Default	log_mime_hdrs off

Synopsis

When enabled, causes Squid to log more information about the request in the access.log file. This causes Squid to also write the request and response MIME headers for every request. These will appear in brackets at the end of each access.log entry.

Arguments

on/off

Enable or disable this process

TAG NAME	useragent_log
Description	Using this tag, you can make Squid to write User-Agent field from HTTP requests to the filename specified in this tag
Build Option	--enable-useragent-log
Usage	useragent_log path/filename
Default	none

Synopsis
By default useragent_log is disabled.

Arguments

path	Path for the useragent log file
filename	File that contains useragent logs

Example(s)
useragent_log /var/logs/usragent.log

TAG NAME	referer_log
Description	Squid will write the Referer field from HTTP requests to the filename specified here
Build Option	--enable-referer-log
Usage	referer_log path/filename
Default	none

Synopsis
By default referer_log is disabled.

Arguments

path	Path for the referer log file
filename	File that contains useragent logs

Example(s)
referer_log /var/logs/referer.log

TAG NAME	pid_filename
Description	Used to define a filename where the process id of Squid is stored
Build Option	Default
Usage	pid_filename path/filename
Default	pid_filename /usr/local/Squid3.0pre3/var/logs/Squid.pid

Synopsis
If you don't want Squid to create this file enter none instead of filename.

Arguments

path	Path for the Squid pid file
filename	File that contains pid of Squid's process

Example(s)
pid_filename /usr/local/Squid.pid

TAG NAME	debug_options
Description	This provides a means to configure all of Squid's various debug sections
Build Option	Default
Usage	debug_options section, level
Default	debug_options ALL,1

Synopsis

Squid's debugging code has been divided into a number of sections, so that if there is a problem in one part of Squid debug logging can be made more verbose for just that section. The magic word "ALL" sets debugging levels for all sections. We recommend normally running with "ALL,1".

Arguments

section	Defines which section's information to be logged
level	Defines debugging levels (0-9)

Example(s)
debug_options ALL, 9

TAG NAME	log_fqdn
Description	Turn this on if you wish to log fully qualified domain names in the access.log
Build Option	Default
Usage	log_fqdn on\|off
Default	log_fqdn off

Synopsis

This configures whether Squid will attempt to resolve the hostname, so the the fully qualified domain name can be logged. This can, in some cases, increase latency of requests.

Arguments

on/off

Enable or disable this process

TAG NAME	client_netmask
Description	Defines what portion of the requesting client IP is logged in the access.log
Build Option	Default
Usage	client_netmask netmask
Default	client_netmask 255.255.255.255

Synopsis

You can make the requesting clients IP to be logged as such or only the network part of the IP alone with the host part being zero. For privacy reasons it is often preferred to only log the network or subnet IP of the client. For example, a netmask of 255.255.255.0 will log the first three octets of the IP, and fill the last octet with a zero.

Arguments

netmask

Clients network mask

Example(s)
client_netmask 255.255.120.200