Import Usage

1. General Information.
2. Usage Filters.
3. Usage Filters Parameters.
4. Usage Filters Installations.
5. Usage Logs.
6. Writing Usage Filters

1. General Information.

RODOPI has the ability to bill for various usage, e.g. web traffic, network traffic, voice over IP phone calls, roaming. In order to bill for the different usage RODOPI has to import information from different types of log files. The following diagram shows how the import  works for web traffic and net traffic usage.


RODOPI comes with a utility that reads some wide spread web logs. It is called SimpleLogFilter.exe and is installed in usage-cgi subdirectory of RODOPI's directory.This utility runs on NT and is capable of parsing log files that are on the same machine on which it runs, or are on other machine (using FTP). This filter can parse Common Log Format, Extended Log Format, IIS log format. It runs on the Web Server as a CGI. When called from RODOPI server it reads the log files and produces XML output, which RODOPI server on its hand parses and stores in the SQL database. Our clients are encouraged to write their own filters and install them to work with RODOPI. In general RODOPI server looks for the filters installed and runs them passing parameters that you have defined. So before you set up a filter you may have to define the parameters that are required for the filter.
For managing the parameters you go to "Filter Parameters" page.

 

2. Usage Filters.

The UsageFilter is CGI application, which generates XML reports. RODOPI calls the UsageFilter in two ways within a session. First call is to retrieve a list of existing logs (or other sources of information). Following calls are to retrieve data from each log. Importing sessions are scheduled in RODOPI. Each call to a UsageFilter comes with two sets of parameters - internal parameters and additional parameters.

 

3. Usage Filters Parameters

On this page you can review usage filters and their parameters. Here you will find a list of the installed filters and their parameters.
You can define additional parameters for your UsageFilter from Administration /Usage Filters /Filter Parameters page. First add new usage filter definition and after this add as many parameters as you wish for this usage filter. The name of the parameter will be used as name in name=value pairs in CGI request to your filter. You are able to add a new parameter for an existing filter by pressing the Add button. You have to specify the name of the parameter as it is expected by the filter. Also you enter a meaningful description. If you want to edit the name or the description of a parameter you press the Edit button next to the parameter.
You can add new filter by clicking on "Add Filter" button. You will be lead to a page where you have to enter the description of the new filter. Also you can edit the filter description by pressing "Edit" button next to the filter description. A new page will appear where you can change the description and save changes or delete this filter.

 

4. Usage Filters Installations

Each filter installation should be defined in Administration /Usage Filters /Filter Installations page.On this page you can review usage filters and their installations. RODOPI comes with two predefined filters. The first is the default one for parsing various web log files, VoIP logs and logs from the Net Traffic Collecting program. The second is for those who are members of RODOPI Club and offer roaming to their clients.
Here you have the following options.

  1. Add a new filter definition by clicking on "Add Filter" button. You will be prompted to enter the description of that filter. Later you can define parameters for that filter.
  2. Edit the description of a filter definition by pressing "Edit" button next to the filter description.
  3. Add locations where the instances of that filter are running. You do this by pressing Add button. You will see a new page where you have to enter the URL of the CGI script. As it was mentioned filters are scripts that run on the web server. For a given filter definition you can have several locations. Imagine that you have a filter that reads web usage log files. You have three servers where you host the web pages of your clients. On each of these servers you have a filters installed. So you have to add three different URLs here (one for each machine).
 

Each instance of the filter can manage several log files. You manage the log files that are processed by pressing "Logs" button. Read below for more info.

 

5. Usage Logs

After the installation is added you should add one or more logs to this installation. Usually the log is URL to directory where log files are created, but you can use this parameter to pass different kind of data to your usage filter. When you enter installation logs and click on Add button you will see two sets of parameters.

First set defines the usage log itself. Directory URL is path (can be file path, network path, ftp location, etc.) to the place where logs reside. If you are using special sources of data you can use this parameter to specify the source. Mask is additional parameter to the URL that can be used to select only part of all logs within the location. Delay is the delay between two importing sessions and next run specifies when the next session starts.

Here you have two approaches for Directory URL.

1. The log can be located on the same machine where the Log Filter runs Note you enter here only the directory, without the file name. Example: .

D:\web_servers\logs\common\

2. It can be on other machine. In this case you have to specify a FTP location of the file. Once again, entering the FTP location do not enter the file name. Example: 

ftp://username:password@machine.domain.com/eb_servers/logs/common/

In the field Mask you specify the file name of the log file. Note that you can put the file name here but you can specify a mask for a number of files to be processed .
Example: If you want a specific file in the above mentioned directory to be processed you just enter its name here 

logfile.txt

However if you want to read a lot of log files of the same type, and all of them are in the directory you enter a mask e.g. 

ex*.log, *.txt, *.*

From Delay combo box you select the delay between two consequent processing of the log file.
In the Next Run field you enter a date or a date and time when the processing of the log file should run next time.

 

Second set of parameters is for additional parameters you defined for this filter. Here you enter values of the parameters as they are defined for that log filter (see Usage Filters Parameters). The values you specify here will be used as value in name=value pairs in CGI request to your filter. It is good idea to specify a parameter for maximum count of records transmitted during one session. This way you will reduce system overheads and improve performance of imports.

RODOPI comes with two predefined filters. The first is the default one for parsing various web log files, VoIP logs and logs from the Net Traffic Collecting program. The second is for those who are members of RODOPI Club and offer roaming to their clients.
Also RODOPI comes with predefined parameters for these filters. The meaning of the parameters for the first one is the following:

And for the roaming filter there is only one parameter called Domain here you enter your roaming ID as your ISP is registered in RODOPI Club. If you are not member of RODOPI Club do not enter nothing here.

If you want to edit the values of these parameters you just click the Edit button.
Use Reload page button when you want to see whether there are changes in the "Next Run" column of the page. It is useful because you can see whether the file was processed or not and fix some parameters of the installation.

Next to each log file there is a button Files. You use it to see a page with a lot of information about the processing of that file. On this page You can get info for the last error message and for the status of the process (whether it is completed or in progress). The Offset column shows where the filter will start from next time it reads the log.

 

6. Writing Usage Filters

You can write your own usage filter. To do that follow the instructions in this document Creating Usage Filters.