For estimation of the available bandwidth in the upstream path just run
For instance, if we know that the file http://www.ics.forth.gr/~papadog/abget/abget-1.0.tar.gz
is large enough for the stream length that we want, we would run abget
for the downstream as follows:
Remember that you have to run abget as root.
Also, you may want to define if you prefer a linear probing algorithm (default) or
a binary search probing algorithm (-l or --linear and -b or --binary respectively).
In the linear probling algorithm we start probing at the rate Rmin and we increase it
using as step the estimation resolution, till the rate Rmax is reached.
In the binary search probing algorithm we always probe at the rate (Rmin+Rmax)/2, and
if the rate is found higher that the actual available bandwidth we continue in the
first half of that range, else we do the same process in the second half.
We stop when we have reached the given estimation resolution or when the variation
range of the available bandwidth is found larger than the given estimation resolution.
You can also set properly some important tool's parameters for the estimation process:
If you need so, you can define the source port (by default a random one free port
is chosen) and the target port (port 80 by default).
A typical output of abget is shown below (while running the linear probing algorithm
for a downstream path with probing range 0 to 100 Mbps, estimation resolution 10 Mbps
and 5 streams per every probing rate):
You can easily edit all the parameters of the tool to the values you want in the
abget configuration file.
These will be the default values, you may also want to change them from the command
line parameters.
gsearch is a crawler that searches for files larger than a given
size in a particular web site. gsearch uses GoogleAPI in order to
search the web for several types of files (pdf, ps, doc, etc)
and returns the url of the files that satisfy the given parameters.
DEPENDENCIES:
The results are printed and also saved to the Crawl_Results file.
3. Running abget
3.1. Starting a measurement
For available bandwidth estimation in the downstream path run
# abget -d -f filename hostname
where hostname is the name (or the IP address) of a web server and filename
a sufficiently large file in that web server.
# abget -u hostname
where hostname is the name (or the IP address) of the web server that you want.
# abget -d -f /~papadog/abget/abget-1.0.tar.gz www.ics.forth.gr
For the upstream path:
# abget -u www.ics.forth.gr
3.2. abget parameters
This was the simplest way to run abget. However, the are some important
properties that you are able to configure before running the tool.
Usage:
abget [-d | --downstream]
[-u | --upstream]
[-b | --binary]
[-l | --linear]
[-p <target-port>]
[-w <source-port>]
[-s <source-ip>]
[-i <network-interface>]
[-r <estimation-range>] (Mbps)
[-x <variation-range>] (Mbps)
[-t <max-time>] (seconds)
[-L <Rmin>] (Mbps)
[-H <Rmax>] (Mbps)
[-P <stream-length>] (packets)
[-N <number-of-streams>]
[-I <idle time>] (microseconds)
[-D <writefile>]
[-c <configuration file>]
[-v | --verbose]
[-h | --help]
[-f <filepath>]
<host>
You can determine if you want to measure the available bandwidth in the
downstream path (default) or in the upstream path (-d or --downstream and
-u or --upstream respectively).
3.3. abget output
While running abget, you can see the probing rates and whether each iteration is
characterized as Increasing (probing rate higher than the available bandwidth),
Non-increasing (less than the available bandwidth) or Grey.
The trend of every stream is decided from the majority of these iterations.
Finally, the tool reports the variation range of the available bandwidth in that time period.
In case that the available bandwidth follows a non-stationary process, which happens if
the path's utilization varies significantly in very small timescales, the abget will report
this state.
:/home/papadog/abget# ./abget -f /packages/pdf/business/26walmart.pdf www.nytimes.com
Configuration file /usr/local/etc/abget/abget.conf loaded
=============== a b g e t ===============
source = 139.91.70.47 [139.91.70.47]
target = www.nytimes.com [199.239.137.200]
target port = 80
source port = 11070
--- Downstream path ---
--- Linear probing ---
Probing from 0.000000 to 100.000000 Mbps with resolution of 10.000000 Mbps
Rate=10.000000 Mbps: NNNNN
Rate=20.000000 Mbps: NNNNN
Rate=30.000000 Mbps: NGNNN
Rate=40.000000 Mbps: NNGNG
Rate=50.000000 Mbps: GGING
Rate=60.000000 Mbps: IGIII
Rate=70.000000 Mbps: IIIII
Rate=80.000000 Mbps: IIIII
Available bandwidth estimated: 40.000000 - 60.000000 Mbps
Execution Time 124.956318 seconds
=========================================
A more detailed output can be choosen using the verbose output (-v or --verbose).
Also, the output can be written to a file (-D filename).
4. Configuring abget parameters
Except from the command line parameters, it is easier to use the abget configuration
file (abget.conf).
After installing abget, the configuration file is located in the installation
directory (e.g. /usr/local/etc/abget/abget.conf).
It also exists locally in the abget folder.
At last, you can load your own configuration file using the -c 5. Finding suitable files in a server
We offer three scripts in order to locate proper files into a given web server.
Usage:
./gsearch.pl -v [-w writefile] [-r readfile] [-s size] [server]
-v: verbose
-w: write results to file <writefile>
-r: read server(s) from file <readfile>
file format: <server>\n
if readfile is not provided then server name is needed
-s: limiting results to files greater than size bytes (default = 100KB)
In order to use GoogleAPI someone must give a license key provided by google.
We provide a valid license key inside gsearch.
If you want to create and use your own key, please visit:
http://code.google.com/apis/soapsearch/ .
gsearch needs SOAP-Lite. Run "./configure" to be sure that it exists in your system.
If this package is not present, then you have to install it.
If you use a linux distribution like debian, then you need to install the
package "libsoap-lite-perl" (e.g. apt-get install libsoap-lite-perl).
Else, you can install it using "perl -MCPAN -e 'install SOAP::Lite'"
(if you already have CPAN.pm installed).
Or you can download it from http://search.cpan.org/.
Usage:
./findfiles.pl [-v] [-s size] [-m number_of_files] [server]
-v: verbose
-s: limiting results to files greater than size bytes (default = 100000)
-m: maximum number of files to return results (default = 3)
Usage: python crawler.py host [depth] [size] [limit]
The default depth (for wget recursive crawling) is 2.
The default size for the suitable files is 20 KB.
The default value for limit (amount of data that wget will download from the web server
before stop) is 1 MB.
6. Contact and additional information
Additional information can be found at http://www.ics.forth.gr/~papadog/abget .
A more detailed description of the tool is given in the paper
"Available bandwidth measurements as simple as running wget", presented in PAM2006
(http://www.ics.forth.gr/~papadog/abget/abget.pdf).
You can also contact the authors by email: papadog at ics.forth.gr, danton at ics.forth.gr, athanat at ics.forth.gr, dovrolis at cc.gatech.edu, markatos at ics.forth.gr