Release notes for DAQ/HLT-I S/W Release tdaq-02-00-00

General notes from s/w librarian

Packages and tags used in the release

Changes in packages (in ABC order)

AccessManager

Policies

The XACML policies generated by the Policy Adminitration Point code use the "Ordered Permit Overrides" policy algorithm instead of "Deny Overrides" policy algorithm.
New resource categories:

• DataBase category for OKS editor and administration tools. 
• ResourceManager category for the Resource Manager TDAQ component.
• BCM category for the BCM subdetector panel. 

Server

• Allow to set the server backlog size through an environment variable TDAQ_AM_SERVER_SOCKET_BACKLOG.
• Added the PAP_UPDATE control message to invalidate the internal PAP cache when the policies are changed.
• Reads the configuration variables from an LDAP server if provided 
• Logging levels naming changed: NORMAL -> MINIMUM, VERBOSE -> NORMAL, VERY VERBOSE -> VERBOSE

Client API

Reads first the configuration from the file /sw/tdaq/AccessManager/cfg/client.cfg, then from environment. If the configuration file is not found, then an information messages is logged to indicate this. The production environment must make sure the file is available (even if it's empty). The configuration file must contain lines with the following format:

#The line with comments must start with '#' character
VARIABLE_NAME=VARIABLE_VALUE

C++ API

• Contact the next AM server in the list in case a timeout occurs during answer receiving from the server. 
• The am server history contains now the server ip addresses next to their names.
• Use poll function for the socket timeout operations. 
• Changed the gethostbyname function with getaddrinfo because the former is not thread safe.

Java API

• Set the log level to none if the AM authorization is disabled so that no log files and directories are created. 
• Fixed the RunControlResource object to get the partition name from the ResourceTypePartition input value

Tools

• amUserRoles:
• added the "-G" flag to specify a role category to prefix the roles to be processed
• allow to process more users in the same time provided either in the command line or in a input text file
• amPAP:
• added the functionality to update the policies on the local disk of AM servers defined in LDAP

clips

Description

Changes in tdaq-02-00-00

New CLIPS version

config

C++ API Changes

Java Changes

DAQPanel

• --setup_dir <setup_dir>        Directory containing setup scripts [default is $HOME]
•  --db_dir <db_dir>                Directory containing database files [default is $HOME]
• --general_dir <general_dir> Directory used to save/get files [default is $HOME]
  

• General tab: Busy Presenter;
• Mon Advanced: MDA Browser, SFO Display, Det Mask;
• Ctrl Advanced: (De)Activate Automatic Recovery.
  

dbe

General description

• DB access by oksconfig (oks xml files), rdbconfig (RDB servers) and roksconfig (DB in Oracle archives) plugin.

• Data preview in table and tree widgets

• Object creating and change

• UNDO history of user change

• Plugin interface with example how to write customized consistency check

• Python interface to run python scripts

• Search by string matching the name of classes or objects or searching by matching UID of objects in DB

• Graphical view of data

• Query builder and standalone find interface

• Local copy saving of DB changes

• Export to HTML

• Open multiple configurations

• Bulk operations over data

• -f file-name    oks databse file name.

dcmessages

Changes since tdaq-01-09-03

• The messages along the trigger path (L2PU-->L2SV-->SFI-->DFM) have been modified following the new version (2.5) of dc-022. The changes were needed to support multiple pROSs in the system and consist in trasporting the pROS NodeID, event by event, from L2PU to the SFI.

DFConfiguration

General changes

Configuration Schema Changes since tdaq-01-09-01

Changes related to EF

• Removed EF_SubFarm class
• use HLTSegemnt instead
• IS/OH server names retrieved from envirnoment variable automatically set by InfrastructureApplication::SegmentProcEnvVarName
  
• Added class for configuration of EFD event sampler (EFD_SamplingTask)
• Added parameters to enable event stripping at EFD output (EFD_OutptTask::eventStripping)
  

Changes related to EB

• Added a fourth value "Preload" to the TriggerSource attribute of the DFMConfiguration class
• Added a relationship to a SFODBConnection object in the SFIConfiguration class

Information Service Info Description Changes since tdaq-01-09-01

dc_is_info

• EFD, added parameters: EventsRecovered, latestGID, EventDuplicWarnings
  
• DFM, added parameters: event duplication warnings, failed assignments
• SFI, added parameters: NumBCIDErrors, EventDuplicWarnings, NumMissingL2Results

ros_is_info

Configuration Data Changes (examples) since tdaq-01-09-01

DFM

Changes since tdaq-01-09-03

• A new triggering mode, "Preloaded", has been added. This is an extended internal mode, in which L1IDs are preloaded from files. The files must be specified in the DFParameters object. If files are not found or are not readable, the DFM will fall-back in internal mode.
• When an assignment message cannot be sent, the corresponding event is reassigned to another SFI without decreasing the event credits.
• The size of the Clear messages can now be set to any value (i.e. even larger than 100). The DFM will never send a clear messages larger than the defined size
• Two new IS variables have been added: number of possibly duplicated events (reassign) and the number of failed assignments

dqm_config

New features:

• DQM.schema.xml changes to accomodate new features:
• geometrical description (layout and shape) of DQRegions and DQParameters to be used by the DQM Display application
• plugins to define DQInput and DQOutput to be used by the dqm_agent application
  

dqmf

New features:

Some changes are required for all DQ configurations !!!

Implementing custom stream

• Declare a class which inherits from the dqm_core::Output interface and implements the publishResult, activate and deactivate functions (empty implementations should be provided for the two addListener functions). The main work is done by the publishResult function which will be called any time a new DQ Result is produced. For example:

class DQFileOutput : public dqm_core::Output
{
	bool 		m_active;
	std::string	m_filename;
	std::ofstream 	m_out;
	
	// Constructor mas have fixed format. The parameters vector
	// contains parameters from the OKS configuration DB
	// Constructor may throw any exception which is based on ers::Issue
	DQFileOutput( const std::vector<std::string> & parameters ) throw ( ers::Issue ) 
		: m_active( false )
		  m_filename( parameters.size() ? parameters[0].s_str() : "" )
		  m_out( m_filename )
	{
		if ( !m_out )
			throw ers::CantOpenFile( m_filename );
	}
	
	void publishResult( const std::string & name, const dqm_core::Result & result ) 
			throw ( dqm_core::Exception )
	{
		if ( m_active )
			m_out << result << " has been produced for the '" << name << "' parameter" << std::endl;
	}
	
	void activate() {
		m_active = true;
	}

	void deactivate() {
		m_active = false;
	}
};

• Create the OutputFactory class for your specific Output and register it with the FactoryManager. One can do this "manually" by declaring a new class which inherits from the dqm_core::OutputFactory and implement the createOutput function. A more simple (and preferred solution) is to use the helper template class and just place a sinlgle line of code into your stream implementation .cxx file, e.g.:

namespace
{
    dqm_core::OutputFactoryT<DQFileOutput> __factory__( "DQFileOutput" );
}

• Compile and build a shared library from your class. Create OKS configuration object of the DQOutput class which describes the DQFileOutput.

<obj class="DQOutput" id="MyFileOutput">
 <attr name="Name" type="string">"DQFileOutput"</attr>
 <attr name="Library" type="string">"lib_my_dq_output.so"</attr>
 <attr name="Parameters" type="string" num="1">
  "/tmp/DQResults.txt"
 </attr>
</obj>

• Last step is to add this object to the DQOutputs relationship of the DQAgent.
  

<obj class="DQAgent" id="Calorimeter">
...
 <rel name="DQInputs" num="1">
  "DQInput" "DefaultInput"
 </rel>
 <rel name="DQOutputs" num="2">
  "DQOutput" "DefaultOutput"
  "DQOutput" "MyFileOutput"
 </rel>
</obj>

dvs

dvs_gui

New package to implement gui for dvs core

• This application (Qt4 implemetation) is supposed to be the replacement of the java implementation of the dvs display.
• It should provide the same functionalities as the old gui.
• New gui is started with:
• dvs_start_gui.sh db_name partition_name
• igui starts the new dvs gui
  

• Options not implemented yet:

1. Combine tests output
2. Save tests output
   
3. Refresh test output panel
   
4. Interactive session
5. Test mask selector
6. Log viewer settings

efd

General changes

• Implemented TCP_KeepAlive option for EFIO communication (see EFIO oks class)
• Finalized flow control implementation, now he can chose among: SizeAlgo, TimeAlgo and OldestTimeAlgo. See EFD_SharedHeap oks class.
  
• Added configuration options for events tagged as "Express"
• Removed RoutingTask (useless)
• Randomized access to request queue. This makes the ExtPT behaving like a FIFO in all the running conditions
• Implemented SharedHeap recovery procedure. If the efd is restarted with the "-recover" command line option, the events contained in the previous sharedHeap file are recovered and forwarded to the SFO (with tag="debug"). The mechanism is configurable via the EFD_SharedHeap::recovery parameter
• Added support for event stripping in outputTask. If the event is only tagged as "Calibration", the vent is stripped according to the list of RoBs hosted in DaqEvent header. This facility can be enabled via EFD_OutputTask::eventStripping parameter
• The name of IS/OH servers are retrieved from environment: $TDAQ_IS_SERVER and $TDAQ_OH_SERVER. N.B.: the EF_SubFarm oks class has been removed
• Added a samplingTask in charge of providing events to the emon infrastructure. Added a correspaonding oks class in the schema 
  
  
  

efio

Changes since tdaq-01-09-03

• EFIO Server

  • Support for the TCP_KEEPALIVE mechanism has been added. The mechanism is turned on and cotrolled by three dedicated parameters of the acceptConnection method: tcp_keep_idle, tcp_keep_int and tcp_keep_cnt. If tcp_keep_idle is zero, TCP_KEEPALIVE is not activated. Individual handlers are notified about the keepalive status

• SFO EFIO Handler

  • The TCPrealAll wrapper class has been removed. We now use directly the TCP class from the transport package. In particular, for message reception, the receive_count methods are used
  • In the TCP_KEEPALIVE mechanism is active, the event reception is performed without timeouts. In fact we rely on TCP_KEEPALIVE for the handling of dead connections

• SFI EFIO Handler

  • A new class SFIEFIOEvent has been added. The aim of this class is to provide a standard data interface to the SFIEFIOHandler, shielding it from the actual data implementation. Moreover the class allows for EFIO information to be returned to the callers (e.g. number of re-sends). The default implementation of the SFIEFIOEvent class includes the daqevent plus the eformat FullEventFragment

• Helper functions

  • efio::getPort: an efio::PortError exception is thrown if the provided application is not either a SFIApplication or a SFOApplication
  • efio::strip: generic stripping function to be used by EFD and SFO. The struct iovevc * &strip_iovec and eformat::write::FullEventFragment * &stripped must be empty pointers. The corresponding resources will be allocated by the function. It is then responsability of the caller deleting them.

emon

General changes

• EMON provides common OKS schema for defining event sampling parameters for the monitoring frameworks used in the TDAQ environment (e.g. GNAM, Athena PT). OKS schema file is called daq/schema/emon.schema.xml and defines 2 abstract classes - SamplingAddress and SelectionCriteria and SelectionParameters class which inherits from both of them.
  

• All applications which are using EMON must be linked with 2 libraries: libemon.so and libemon-dal.so.

C++ API changes

New Monitoring task API

• Both emon::SamplingAddress and emon::SelectionCriteria classes provide constructors taking IPCPartition and emon::dal::SelectionParametrs objects as parameters. This is the easiest way of creating them.
  

• There are 3 possible ways of creating an emon::SamplingAddress without using OKS configuration:

• SamplingAddress( const std::string & type, const std::string & name );

• SamplingAddress( const std::string & type, const std::vector<std::string> & names );

• SamplingAddress( const std::string & type, unsigned char number );

• Creating an instance of emon::SelectionCriteria manually (not by reading it from the OKS configuration) is now a bit tricky. Here is an example:

unsigned int lvl1_type = ...;

std::vector<unsigned char> lvl1_bits;
lvl1_bits.push_back( 13 );
lvl1_bits.push_back( 131 );

std::string stream_type = "physics";

std::vector<std::string> stream_names;
stream_names.push_back( "electron" );
stream_names.push_back( "photon" );

emon::SelectionCriteria criteria(
	lvl1_type,
	emon::SmartBitValue( lvl1_bits, emon::logic::AND, emon::origin::BEFORE_VETO ),
	emon::SmartStreamValue( stream_type, stream_names, emon::logic::OR ),
	emon::StatusWord( ) );

• Global emon::select function has been removed. Now in order to start receiving events one has to create a new instance of emon::EventIterator class.

   EventIterator(    const IPCPartition & partition,
                     const SamplingAddress & address,
                     const SelectionCriteria & criteria,
                     size_t buffer_size = 100,
                     bool dispersion = false ) throw ( emon::Exception );

  
   EventIterator(    const IPCPartition & partition,
                     const emon::dal::SamplingParameters & params )
                                   throw ( emon::Exception );

New Event Sampler API

• Virtual functions startSampling of both PushSamplingFactory and PullSamplingFactory classes do not have anymore emon::SamplingAddress as their first parameter. 

• An emon::SelectionCriteria object which is passed to those functions as parameter has the following structure:
  

m_lvl1_trigger_type.
	m_value (unsigned char);
	m_ignore (bool);
m_lvl1_trigger_bits.
	m_bit_positions (std::vector<unsigned char>)
	m_logic (enum emon::Logic { emon::logic::AND, emon::logic::OR, emon::logic::IGNORE })
	m_origin (enum emon::Origin { emon::origin::BEFORE_PRESCALE, emon::origin::AFTER_PRESCALE, emon::origin::AFTER_VETO } )
m_stream_tags.
	m_type (std::string);
	m_names (std::vector<std::string>)
	m_logic (enum emon::Logic { emon::logic::AND, emon::logic::OR, emon::logic::IGNORE })
m_status_word.
	m_value (unsigned int);
	m_ignore (bool);

gatherer

The parameters of type string 'SumAlgorithm' and 'SumOrAverage' were substituted with enum 'SummingMode' and 'ResultContent' respectively. For the range of parameters see the schema.

genconfig

C++ API Changes

Integer Types

Generated Enum Values

const TestModule& obj;
if(obj.get_Frequency() == TestModule::Frequency::Low) { ... }
else if(obj.get_Frequency() == TestModule::Frequency::High) { ... }
else if(obj.get_Frequency() == TestModule::Frequency::Default) { ... }
else ...

const TestModule& obj;
if(obj.get_Frequency() == "low") { ... }
else if(obj.get_Frequency() == "High") { ... }
else if(obj.get_Frequency() == "default") { ... }
else ...

• reduce chances of various typos,
• be informed during compilation if used enumeration value was removed (i.e. get compilation error instead of run-time error or wrong program behavior).

• names of attribute and valid values are capitalized
• names of attribute and valid values started from non-alpha characted (e.g. digit) is prefixed by undescore symbol
  

 /**
  *  Valid enumeration values for get and set "Frequency" attribute methods.
  */

struct Frequency {
  static const std::string Default;
  static const std::string Low;
  static const std::string High;
  static const std::string _1KHz;
  static const std::string _5KHz;
};

Java Changes

gnam

Changes from tdaq-01-09-01 to tdaq-02-00-00

New configuration schema

1. The gnam schema has been changed following the new emon APIs. The GnamSampler class has been deleted, since the same configuration information is now prodided by the SamplingParameters class defined by emon. Thereby, the GnamApplication class now requires a SamplingParameters instances in the SamplerSpecification relationship. Moreover the SamplerBufferSize attribute has been deleted, since the information is carried by the SamplingParameters class.

GnamHisto changes

1. Added a new wrapper method:
   • const char *GetTitle (void) const

igui

Introduction

General changes  

• The following changes have been done for the IguiPanel interface:
• Add Run Control state as a parameter for connectInfrastructure(String state) method
  
• Add old Run Control state as a parameter for runStateChanged(String oldState, String newState) method
• Add a new method (String elogString()) to allow to insert user panel specific information in the elog entry
• The following changes have been implemented in the interface with the Resource Manager:
• The default mode for an IGUI is Status Display
• The option useRM has been added; if the igui_start uses the option -DuseRM, IGUI tries to obtain the highest control level according to the resources available in Resource Manager and to the user role in Access Manager
• IGUI is using now the requestResourceForMyProcess method to obtain resources from Resource Manager. This allow to free the RM resources at the igui_start script exit.
  
• The folowing changes have been implemented in the IGUI - ELOG interface:
• move the TTC partition list from an attribute to the message text
• add HLT Prescale Key in the automatic e-log entry, in the message text
• add Project Tag in the automatic e-log entry, in the message text
  
• The Statistics sub-panel  in the Monitor panel has been updated taking into account the emon changes in the statistics information published in the Monitoring IS server (information names, information structures).
• For the event counters and rates displayed in the IGUI, the sources of information are taken from the new configuration database classes IS_InformationSources and IS_EventsAndRates.
  
• A RunParams confirmation dialog has been added after the Start command. For the moment the user is requested to check the Beam Type, the Recording option and the Project Tag.
  
• The button get wait condition has been added. The button is enabled if the application selected in the RC tree is alive, is a controller and is busy. As a response to the command sent using this button (CMD_CONDITION command), the controller in the busy state sends a message (with severity INFORMATION) listing the children that it is waiting for.
• For mrs_monitor the following feature requests have been implemented:
• Add Pause and Continue buttons (at Pause the mrs_monitor subscription to MRS is removed, at Continue the subscription is done again)
  
• Add default number of displayed messages as a command line option (-n messages-number)
  

• Try to improve the re-connection of an IGUI in Status Display mode to the new infrastructure if there are problems restarting the partition.
  
• In the MRS table, insert a space between text and pid in the long messages.
  
  

Igui

• A panel to send transition commands to the RootController;
• A panel containing the run control tree;
• A panel to enable or disable segments and resources.
  

ispy

The ispy package is a Python interface to IPC, the Information Service (IS) and the Online Histogramming Service (OH). It allows easy access from a scripting language to all the information in IS and OH.

To use the package, simply start the tdaq_python interpreter:

    % tdaq_python
    Python 2.5 (r25:51908, Oct 18 2007, 16:26:11)
    [GCC 3.4.6 20060404 (Red Hat 3.4.6-8)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import ipc
    >>> import ispy
    >>> import oh

The following classes from IS are available in Python:

• IPCPartition
• ISCriteria
• ISInfo
• ISInfoAny
• ISServerIterator
• ISInfoIterator
• ISInfoDictionary
• ISInfoAttribute
• ISType
• ISObject (a more dynamic version of ISNamedInfo)

The following minimal script shows how to use them in a simple way:

#!/usr/bin/env tdaq_python

part_name = 'rhauser_test'

import sys
from ipc import IPCPartition
from ispy import *

p = IPCPartition(part_name)

if not p.isValid():
    print "Partition:",p.name(),"is not valid"
    sys.exit(1)

x = ISObject(p, 'DF.L2SV-1', 'L2SV')
print "Exists: ",x.isExist()

x.checkout()
print x

sum = ISObject(p,'DF.L2SV-Summary', 'L2SV')
sum.LVL2_events = x.LVL2_events
sum.checkin(False)

Several utility classes are provide to make it easier to read arbitrary information from IS. See the examples directory for how to use them.

The ispygui.py is a simple application using Tcl/Tk to give an overview of the dataflow part of a running partition.

The callback routines can now be used in separate python threads, the actual code is protected and deals nicely with the interpreter.

The following classes are available from the OH package. Note that many methods are pythonized in such a way that the user does not have to use a method that is specific for TH1, TGraph etc.

• getRootObject(partition, server, provider, histoname) (free function).
• OHRootReceiver (override single receive() method).
• OHRootProvider (single publish() method).
• OHSubscriber

ls

Introduction

This package substitues the obsolete logService. A new requirement whereby database technologies other than ORACLE had to be dropped came in around spiring 2007. This new requirement meant the re-writing of the logService package, which was higly dependent on MySQL. This opportunity was taken to refactorize the code, especially the log manager, which was never very user friendly. The database access in C++ is done using the CORAL interface, which hides the underlying technology. For the log manager, JAVA was the language chosen, since it brings in the flexibility requiered to make this tool more intuitive. The resulting java application can be run from the console, or remotely using the Java Web Start technology.

Known issues/bugs

In the Log Manager, when righ-clicking on the Messages Table (right hand side panel) or Partition Table (left hand side panel) a Refresh button pops up. This is used to update the current view of partitions and messages, respectively. Refreshing these objects only works if one right-clicks the mouse without moving it; that is, press and release without changing the X and Y pointer of the mouse. This may be a problem with Java. In the meantime, one can also perform a Refresh via the options in the top menu File: 'Refresh tree' and 'Refresh table'.

To be implemented

Add an option to display statistics, internal and from IS.

Changes from previous release

The layout of the Log Manager has slightly changed. There are two distinctive search criterias now. One related to the Log Message table (the same as in the previous release) and a new search associated to the Partition Tree (left panel). This new search allows users to find a node, be it a Partition or Partition and User, among the hundreds that form the left hand side Tree panel. The Log Manager expands this node and makes it visible, thus facilitaing the navigation through this tree. One can also enter a run number - the Partition and User are not needed but they can also be specified - which will expand the Parition.User node and display the corresponding Log Messages in the table.

Migrate to the deSEALed version of CORAL.

Example applications

Applications

Log Manager

Usage: log_manager 

Instruction on how to use to be written.

The Log Manager can also be run using Java Web Start technology from the link:

If possible, it is preferrable to launch this tool as a Java application rather than from the link above.

lsReceiver

Description: This application subscribes to the MRS service to receive and log on a database messages produced by TDAQ applications.

Usage: lsReceiver [-p partition-name] [-u user-name] [-n IS-server-name] [-s threshold-size] -c connect-string [-S subscribe-expression]

Options/Arguments:

        -p partitionName     Partition name

        -u userName          User name

        -n ISserverName      Name of the Information Service to publish the message rate into.

        -c connectionString  Database connection string. 

Test units

logTest

Description: Test binary for the Log Receiver application.

Usage: logTest -c connect-string [-p partition-name] [-l complexity-level]

Options/Arguments:

        -p partitionName      Partition name

        -l level           Level of Complexity of the test [1: open/close - 2: tests the Log Service Infrastructure].

        -c connectionString  Database connection string. 

Utilities

logSelect

Description: Application to retrieve log messages for a given partition according to the search criteria specified. By default, messages are dumped on std::cout.

Usage: logSelect -c connect-string -p partition-name [-i message-name] [-m machine-name] 

                 [-a application-name] [-l time-low] [-u time-up] [-s severity]  

                 [-x text] [-r parameters] [-d order-list] [-e max-rows] [-f offset-row]

Options/Arguments:

        -p partitionName      Partition name

        -c connectionString   Database connection string. 

        -i message-name       Message name or ID.

        -m machine-name       Machine name where the message was issued.

        -a application-name   Application name where the message was issued.

        -l time-low           Lower time threshold.

        -u time-up            Upper time threshold.

        -s severity           Message severity:

                                 1 - FATAL

                                 2 - ERROR

                                 3 - WARNING

                                 4 - DEBUG

                                 5 - INFORMATION

                                 6 - SUCCESS

        -x text               Text in the message body.

        -r parameters         Message parameters.

        -d order-list         Parameter to sort the messages by.

        -e max-rows           Maximum number of rows to retrieve from the database; 100 by default. If 0, all entries are retrieved.

        -f offset-row         Offset in the table to retrieve the messages from.

logDelete

Description: Application to remove log messages for a given partition according to the search criteria specified.

Usage: logDelete -c connect-string -p partition-name [-i message-name] [-m machine-name] 

                 [-a application-name] [-l time-low] [-u time-up] [-s severity]  

                 [-x text] [-r parameters]

Options/Arguments:

        -p partitionName      Partition name

        -c connectionString   Database connection string. 

        -i message-name       Message name or ID.

        -m machine-name       Machine name where the message was issued.

        -a application-name   Application name where the message was issued.

        -l time-low           Lower time threshold.

        -u time-up            Upper time threshold.

        -s severity           Message severity:

                                 1 - FATAL

                                 2 - ERROR

                                 3 - WARNING

                                 4 - DEBUG

                                 5 - INFORMATION

                                 6 - SUCCESS

        -x text               Text in the message body.

        -r parameters         Message parameters.

logGetPartitionNames

Description: Application to retrieve the list of partition names.

Usage: logGetPartitionNames -c connectionString 

Options/Arguments:

        -c connectionString  Database connection string. 

logCleanDatabase

Description: Application to clean the database by removing all the existing tables.

Usage: logCleanDatabase -c connectionString 

Options/Arguments:

        -c connectionString  Database connection string. 

mda_browser

General features

• List of archived TDAQ Partition names
• List of archived Run Numbers
• List of archived histogram providers
• List of archived folders

1. If this histogram is available in the file which has already been downloaded from CASTOR then it will be immediately displayed
2. Otherwise if the browser is running in ATCN (on a Point 1 machine) then it will try to get this histogram from the COCA cache directory
3. If all of the above failed then a user will see a pop-up message asking whether he/she wants to download the necessary file from CASTOR. If the "Yes" button is pressed then download is started in the background and it's possible to continue browsing the archive content. As soon as download is finished the new ROOT Canvas window will be created with the requested histogram. In order to monitor the status of the CASTOR files download one can switch on the Download Monitor window by pressing the last button in the browser's tool bar.
   

MonGatherer

Changed the schema.

mrs

General changes

• -p partition-name    partition name.

• -c subscribe-expression   subscribe expression.

• -s use PMG synchronization.

oks

C++ API Changes

OKS Server

OKS Data Editor Changes

• Add search by file name in the main window
• Add search by class and object ID in the Data File dialog
• Add group file operations from the File menu:
• Save all updated files (<Ctrl-S> shortcut)
• OKS server related operations:
  
• Release User Repository (<Ctrl-D> shortcut)
• Update User Repository (<Ctrl-U> shortcut)
• Commit User Repository (<Ctrl-C> shortcut)
• Add "Referenced By" function from Object dialog (available on right mouse click from dialog's icon)
  

oks2coral

• Migrate to deSEALed version of CORAL.

oksconfig

• Migrate to deSEALed version of CORAL (roksconfig plugin).
  
• Follow config changes to support 64-bits platforms.
  

OMD

Changes

• Support for vector fields in IS objects is reintroduced and can be enabled from Subscription Editor.
• There are two small applets in the status bar, an animated icon showing updates, another showing the partition status and run number. Partition status widget is under testing
• Cleaned up the menus, removing unused actions.

OnlineRecovery

Introduction

RunController expert system

Server

Core functionality

• An application with decision set to RESTART will be restarted up to a maximum of 5 times. After this restarts will not happen and it will be considered an error instead. If the last restart happened more than 30 min ago the counter is reset (it is also reset if the controller goes back to NONE state).
• An application with a failed test will be re-tested up to a maximum of 3 times. After this a failed test is considered an error. If the last retest was more than 10 min ago the counter is reset.
• If specific recovery is defined (through extra rules files) these might override the database settings.
• An application with decision IGNORE will not be ignored if they are depended upon by an application that is running and has membership IN. In this case it is considered and error and an ers message will be sent containing all the applications depending upon it.

Expert system server

• State - if false no recovery is done.Otherwise recovery is performed according to the following settings.
• Ignore Xoff - if true ROS XOff is not taken into account, otherwise recovery is not done if ROS Xoff is the cause of busy ROD.
• Automatic - if false the user is asked before recovery is performed. If true recovery is done without user input.

Changes since tdaq-01-09-01

• Automatic disabling on dbproxy failure
• Automatic disabling of PTs if their EFD is disabled by OnlineRecovery
• In specialized recovery applications are now terminated directly (no longer attempting to cycle them down nicely)

Known bugs

Utilities

error_viewer 

PmgGui

• Fixed null pointer exception in button event handler;
• Adapted to the IS process info name change done in the ProcessManager package;
• Fixed small EDT violation.
  

ProcessManager

• IS process information name has been modified to make process search simple and more efficient. It now includes the name of the host where the process is started:  <host>|<process name>;
• Added some additional verbosity when the AccessManager reports some error.

• pmg_start_app: Added option to wait for the process to exit; two more options are available:

PTIO

Introduction

Changes since tdaq-01-09-01

• ptioInterface: added new parameter in getEvent() method in order to forward DaqEvent pointer. The parameter has a default value (NULL), so the interface change is transparent for other packages.
• ptioEfd: added a configuration switch (ptioEfd::eventChangePolicy) that permits very specific configuration characterized by the use of 2 ExtPTsTask in series. Indeed, if the value is set to "Flat" it triggers the serialization of the modified event in a contiguous chunk of memory. However in all the other scenarios, the default behaviour (that keeps the data scattered in 2 different blocks) must be used.
  

rcc_time_stamp

Introduction

General changes since release tdaq-01-08-00

• Linux usleep added to ts_wait for long durations. This shoudl save some CPU time
• Some of the issues with 64-bit platforms fixed. Final tests pending
  

Changes in API

Known bugs, problems and limitations

RCDLtp

Introduction

General changes

RCDLtpi

Introduction

General changes

RCDLtpiModule

Introduction

General changes

• no changes

RCDLTPModule

Introduction

General changes

• added new class: LTPCTPParasiteConfig to be used when running in parasitic mode (LTP is master, but still uses the timing signals from the CTP)
• moved setting of the run mode of the pattern generator into connect() transition in order allow to set up other boards that need to be synchronised with the LTP

rdbconfig

• Follow config changes to support 64-bits platforms.
  

rn

• Migrate to deSEALed version of CORAL.
• Add close() method to put final information on run stop and to avoid possible exception in the destructor.
• Always disconnect with database at destructor and re-connect in the close() method.
  

RODBusy

Introduction

General changes

RODBusyModule

Introduction

General changes

• no changes

ROSCore

Introduction

Changes since tdaq-01-09-01

• SequentialDataChannel & SequentialInputHandler:
• Use new ROSHandler interface
  

ROSEventFragment

General changes since release tdaq-01-09-00

• Add exception specific clone and get_uid (by Gordon)
• RODFragment and FullFragment classes updated to new emon::SelectionCriteria API
  

Changes in API

• None

Known bugs, problems and limitations

• None
  

ROSInterruptScheduler

Introduction

Changes with respect to release tdaq-01-09-00

• Globally:
  
• Use new ROSHandler interface (implemented by Gordon)
• RobinInterruptCatcher:
  
• Some bug fixes (mostly what already was available as a patch for tdaq-01-09-01)
• error messages clarified
• InterruptCatcher:
  
• Replace cout by DEBUG_TEXT
• signal handler will only be installed (and remove) if the thread gets started
• ScheduledUserAction:
• getAge() added (required for CPU time-economic scheduling)
  
• UserActionScheduler:
• More CPU time-economic task scheduling
  

ROSIO

Changes since tdaq-01-09-00

• Nothing from M. Joos
  

ROSModules

Introduction

General changes since release TDAQ-01-09-00

• RobinReadoutModule / RobinDataChannel: Support for stopless recovery
• RobinReadoutModule:
• Update for new RobinInterruptCatcher interface
  

ROSRCDdrivers

Introduction

General changes since release tdaq-01-09-00

• Globally: SLC5 compilation errors fixed
• robin:
• partial redesign of the ISR for better IRQ handling (only what already existed as a patch for tdaq-01-09-01)
  

Changes in API

Known bugs, problems and limitations

ROSRobin

Introduction

General changes since release tdaq-01-09-00

• configureDiscardMode added. This mode allows disabling or re-enabling a ROL on the fly (for the "stopless recovery")
• Bug fixes for the interrupt handling (only what already existed as a patch for tdaq-01-09-01)
  

Changes in API

• Rol.cpp:  method configureDiscardMode() added

Known bugs, problems and limitations

RunController

Introduction

This package contains the Run Control for the TDAQ. It provides two C++ interfaces which allow the user to introduce case-specific actions carried out when a command is received: UserRoutines.h and Controllable.h. The two interfaces have completely different purposes!
A developer shall extend from UserRoutines.h only if he is customizing a controller at an intermediate level of the run control tree (i.e. with child applications). He can create UserRoutines objects to be called at the corresponding state-transitions and commands before and/or after they are processed by the Controller itself (i.e. transmitted to the children).

A developer of leaf applications (run control applications without children) shall only extend the Controllable.h interface. He creates a Controllable object to be called at the corresponding state-transitions and commands.

Changes from previous release

The major change with respect to tdaq-01-09-01 sees the rc_setup as a standalone binary completly decoupled from the RunController framework. This makes for a more robust application that can better recover or output information in case of errors. It should now be easier to understand and maintain the code.

Also important is the migration of the initial PMG and HW testing from the Online Recovery to the RunController. It is now the latter that drives this initial testing procedure and reports to the Online Recovery accordingly.

Other changes refer to bug fixes.

Migration to the new Run Controller

1. In the requirements file (only applicable to TDAQ packages), remove the statement 'use ClipsServer' and/or 'use RunControl' and add instead 'use RunController'
2. The namespace for all run control classes is now daq::rc::, while it was a mixture of RC::, CS:: and rc:: in the past.
3. The old Run Control headers must be replaced as follows:
   1. RunControl/Controllable.h --> RunController/Controllable.h
   2. RunControl/ItemCtrl.h --> RunController/ItemCtrl.h
   3. rc/UserRoutines.h --> RunController/UserRoutines.h
   4. rc/Controller.h --> RunController/Controller.h
   5. RunControl/UserExceptions.h --> RunController/UserExceptions.h
4. In the databases, the Binary rc_empty_controller must be replaced by run_controller. The script RCUtils: old2newRC.sh is available to perform this action automatically.
5. The Constructor of the Controller class has changed. The new constructor needs the following parameters:
   Controller (const std::string & partitionName, const std::string & segmentName, const std::string & parentName, const std::string &expertSystemRules, UserRoutines * preUserRoutines, const std::string & postUserRoutinesLib)
   
6. The API of the UserRoutines class has changed:
1. dropped controller method
   
2. dropped send method
   
3. dropped raiseError method (and ers::fatal message automatically causes the controller to go into ERROR state)
4. mout method dropped
5. mrsReceiver method dropped
6. noAction method dropped
7. resetAction dropped
8. clearError replaced by clearAction
   
9. runParameter and updateRunParameter methods replaced by getRunParameter.
   

Dealing with substates

Substates can be introduced into an intermediate Controller by extending the UserRoutines class and passing it to the Controller as a plug-in. More on this below. Substates are performed when the FSM transition they are associated to is 'finished'. Before the Controller transitions to this FSM state it performs all the substates (there can one on or more) by broadcasting them to the children and waiting for these to acknowledge the termination of a given state. Only when all the substates have been perfromed, does the Controller tranistion to the original FSM state.
For leaf applications substates are dealt with as before, as special types of user commands.

Operation

The Root Controller is now in charge of doing the bootsrapping of the system. When launching the setup_daq script, rc_setup is invoked to start the pmsgerver, IPC server, RDB server and the IS servers: Setup and RunCtrl. Once these applications are running, rc_setup exits, and the setup_daq script launches the actual Root Controller. This two-step operation is necessary since the Root Controller requires of the above mentioned applications to start running the remaining Online Segment infrastructure applications. The main reason in fact is that rc_setup reads the database configuration from OKS (since there is not RDB server running). The Root Controller must however access and subscribe to the database via RDB, and logically can only do so if this service is running beforehand. The Setup and RunCtrl IS servers are required to publish the status, state, test result, etc. of the applications being controlled by the Root Controller and any other controller. This information is read by the IGUI to update the inforamtion displayed to the user. With this new philosphy, the setup server has been removed completly from the TDAQ system.

Each controller is in charge of starting, stopping, distributing commands to and reacting to errors from its children (other controllers or applications). If a Controller dies and is restarted, it goes into the state of its children (if they are all in a consistent state); if there are no children, it goes through all required state transitions in order to reach the state of the parent. This mimics the behaviour of applications that extend from the Controllable interface.

Errors

The RunController hardly takes any decisions when errors occurred. Instead, the corresponding error are forwarded to the Expert System (Online Recovery) which takes a decision accordingly. This decision currently depends on the settings defined on the configuration database. Three fields are available to the user to define the behaviour: IfFails, IfDies, IfError. Note on restart: If the option restart is selected for any of these fields and the restart fails repeatedly, it will cause an error instead.

Timeouts

Substates

Substates are performed before the Controller transitions to a given FSM state, that is, when all its children have successfully moved to that state. Before the Controller actually transitions, it propagates the user-defined substates to its children. When all these perform the action associated to the substate, the Controller will either propagate the next substate if any, or terminate the original FSM transition.

Substating is possible by extending the UserRoutines class. The user must generate a library and pass it to the Controller via the command line argument -u. This facilitates the development of the substates logic as there is no need to recompile the controller code and allows the loading of any substate plugin by simply changing the Parameters field of the Controller in the configuration database.

The example RunController/examples/ExampleSubstate.cc can be taken as a starting point. In order to define substates for, let's say, transition Configure, the user must extend daq::rc::UserRoutines::configureAction as follows:

  bool MySubstates::configureAction()
  {
     if (true == currentSubstate_.empty()) {
        currentSubstate_ = "configureSubstate_1";
     }
     else if (0 == currentSubstate_.compare("configureSubstate_1")) {
        currentSubstate_ = "configureSubstate_2";
     }   
     else {
        currentSubstate_ = "";
     }

     doSubstate(currentSubstate_);     
   
     return true;
  }

In this example MySubstates inherits from daq::rc::UserRoutines. The user must keep track of which substate is currently being handled and the order of substates if there are more than one. In the example above, we use the string currentSubstate_. The method daq::rc::UserRoutines::doSubstate(const std::string& substate) is the interface to the Run Controller to pass the next substate name. When the last substate is processed, the user must call doSubstate("") with an empty string. This informs the Controller that substating is finished.

Known issues/bugs

None

To be implemented

The Controller that launches the initial applications in the rc_setup binary is reactive to the Online Recovery. In other words, it waits for the latter to 'tell it' what to do: start, test, exit, etc. However, the Expert System should be reactive and not active. Therefore, it should be the controller in the rc_setup the one that drives this initial bootstrapping, updating the Online Recovery as necessary.

Example applications

Applications

rc_setup

Description: Binary to start the basic infrastructure needed by the Root Controller.

Options/Arguments:

        -p partition      Name of the IPC partition (default $TDAQ_PARTITION)

        -d database       Name of the database (TDAQ_DB)

        -s segmentname    Name of the segment

        -n controller     Name of the controller

        -R expertSystem   Expert System library name (without the 'lib' prefix or the '.so' extension, followed by the arguments if any.

run_controller

Description: The RunController is a general purpose control entity for the ATLAS Online infrastructure.

Options/Arguments:

        -p partition      Name of the IPC partition (default $TDAQ_PARTITION)

        -P parentname     Name of the parent

        -s segmentname    Name of the segment

        -n controller     Name of the controller

        -u substates      Library with the substates definition. The library name has to be given without the 'lib' prefix nor the '.so' extension.

        -R expertSystem   Expert System library name (without the 'lib' prefix or the '.so' extension, followed by the arguments if any.

rc_test_controller

Description: Test unit for rc_controller binaries.

Options/Arguments:

        -p partition    IPC partition name (default $TDAQ_PARTITION)

        -n controller   Controller names

SFI

Changes since tdaq-01-09-03

• Monitoring event selection

  • The event selection for monitoring purposes has been updated following the new emon selection criteria. In particular one can now select on:
    1. Status word
    2. L1 Trigger Type
    3. Stream Tags
    4. L1 Items (before prescale, after prescale, after veto)
    
    The old magic values, like -1,-2,-3 for the status word, have been removed, since emon now provides a ignore flag for each selection item. The ignore flags are respected by the SFI (i.e. ignored criteria always matchs).

• Multiple pROS support

  • The support for multiple pROS has been implemented. For each event, the SFI expects to receive the NodeID of the pROS holding the L2Result via the assignment message. If the received NodeID is zero, the pROS request is silently skipped. In the received NodeID in unknown, a warning is issued and the pROS request is skipped. In both cases an IS counter, 'NumMissingL2Results', is increased.

• BCID Check

  • The BCID check is now always performed and a dedicated IS counter is increased in case of errors ('NumBCIDErrors'); The configuration switch in the database ('BCIDcheckEnabled') only controls the SFI verbosity in case of detected BCID errors.
    

• SFO-Tier0 Handshake database

  • The SFI is now able to register files into the SFO-Tier0 Handshake database. In order to activate this functionality, a proper SFODBConnection object should be linked in the SFIConfiguration.
    Currently the SFI do not provide any streaming capabilities, therefore the produced files will look like:
    project_tag.runnr.No_Streaming.daq.RAW._lblbnr._AppName._filenr.data

SFO

Changes since tdaq-01-09-03

SFO-Tier0 handshake DB

• The SFOTZ is now a standalone library which provides all the SFO-Tier0 functionalities:
• Integrated DataWriter callbacks
• Database handler thread which provide a single antry point for any database activity

SFO Application

• Events with an empty stream name field are written into files with a "NoName" stream name. In fact, at the file level a stream name is always needed in order to preserve the database consistency
• Stripping functionality has been implemented. Events with non-empty ROB and/or SubDetector lists and a calibration tag are stripped following the received lists. The stripped copy is written into the calibration files, while full copies are written in other file types (physics, express, ...).
• New histograms have been added:
  • Event Size vs. Streamtype_Streamname, filled after the SFO stream processing and stripping ("/SHIFT/SFO/EventSizePerFileStream").
  • Stream correlation after the SFO stream processing ("/SHIFT/SFO/FileStreamCorrelation")
• Do not check anymore for event tagged physics_express since the express stream is now a defined type, which will always be streamed inclusively
• The workaroud for events with multiple tags, including a debug tag, has been removed, since this is now configurable in the steering. See bug 41448

tmgr

Changes since tdaq-01-09-xx

Fixes and improvements

New features

Known bugs, problems and limitations

Changes forseen for next major release

training

Introduction

General changes

• The sampler and monitoring task applications in the Event Monitoring exercise have been updated according to the changes in emon package.
  
• The get_eventcount application in the Controller exercise has been modified to follow database schema changes on event counter information source.
• The ROD Crate DAQ exercise has been removed from the training package (developers of the RCD will use directly the RCD users' guide).
• Some database files have been updated according to the schema changes.

• Update some database files according to the schema changes.

TriP

General changes in tdaq-02-00-00

• Memory consumption reduced by factor of less than 4.
• Fixed memory leaks.
• History depth for L1 longer, 120 points.
• L1 Trigger Items list widget responsiveness fixed.
• Added current time on L1 graphs.
• Butons color on HLT chains put back to Red, instead of DarkRed.

Changes for the user

• None.

To be implemented

• Total change in structure under way, with the new project name - Trigger Rate Presenter.

TTCviModule

Introduction

General changes

• changed attribute "Retransmit" to "Retransmission"

vme_rcc

Introduction

General changes since release tdaq-01-09-00:

• Some issues for 64-bit kernels fixed. Not yet tested.
  

Changes in API:

Known bugs, problems and limitations:

wmi

Introduction

New features

• Added mode of WMI which don not use fork()
• Remove from WMI DQMF plug-in all partitions wich end of name "_mirror".
• Added support frame.
• Not write attributes cellpadding and cellspacing if these values are 0 (in Table class).
• Plot implementation use java.
• Accelerated copy files to WEB server.