ibdm-ibnl-file(1)
NAME
ibnl - a generic IB netlist format
DESCRIPTION
IBDM topology file ibdm-topo-file provide means to describe the IB fabric using a set of predefined systems. A system definition is provided in a single file in IBNL format that describes the internal InfiniBand connectivity of the system in terms of boards and devices. When IBDM starts it parses all the available system definition files before it handles the topology file. The files are located in the following directory relative to the installation prefix: <prefix>/lib/ibdm1.0/ibnl. This man page describes the IBNL file format used to define an arbitrary IB system internal IB connectivity. It outlines the main concepts used by the file, provides details for how to write such a file and provides a formal definition of the file syntax in BNF like format (YACC/Bison readable).
1. Main Concepts
The described file format for defining generic system IB connectivity
uses the following terminology:
NODE
The instantiation of an IB device may it be a switch or a channel
adapter
SUB-SYSTEM
The instantiation of a board or module in a system
SYSTEM
A definition of a board or a module
TOPSYSTEM
The top most system defined in the given file
SUB-SYSTEM MODIFIER
Many systems allow multiple variations of their subsystem such as a
12X version of a leaf board in a MTS9600. The modifier is a suffix
to the board name. The IBNL format supports assigning multiple
names to the same board.
2. File Format
The exact file format is provided in section 4. This section only provides extra information regarding the specific semantics of various sections. The IBNL file is line sensitive as it requires different sections to start on a new line. The file is broken into several SYSTEM (optional) and one TOPSYSTEM sections. Each has one or more names. Connections are defined inside the SYSTEM/TOPSYSTEM section only and thus might be of two types: 1. Between any node or sub-system to other node or sub-system 2. From any node or sub-system to a connector of the system. NOTE-1: The top system can define ports that might be redundant and not connected to any lower level board connector. In these cases the ports are simply omitted from the resulting system. This feature enables defining the front panel ports of a 12X/4X such that if a 12X leaf is selected only the 12X front panel ports are used. NOTE-2: Port width and speed provided at the lowest level have precedence over definitions provided at upper levels of the hierarchy.
3. Guidelines for writing a System IBNL File
The following procedure should be followed in order to provide a new
system IBNL:
a) Name the file after the system name: <any sys name (no
spaces)>.ibnl
b) Define a SYSTEM section for each board included in the system
c) The port names of the boards are just strings, we use the simple
format of Pn where N is just a serial number but you can pick any
name you want. Just make sure it is unique.
d) When different flavors of the boards exists like when you have a 4x
and 12x option for a board name the optional boards with a modifier
postfix. An example for such system that supports a LEAF board of
12x and 4x types would be to define two SYSTEMS:
SYSTEM LEAF,LEAF:4x,LEAF:4X
...
SYSTEM LEAF:12x,LEAF:12X
...
NOTE-3: The instantiations of the LEAF boards in another board
or the top system need not specify the postfix and are being
decided upon only when the topology file is being parsed. The
"SYSTEM LEAF" with no postfix will be used by default. To
continue the example here is how the LEAFs are instantiated in
the top system:
TOPSYSTEM MyIbSystem
LEAF leaf1
...
LEAF leaf2
...
The actual 4x or 12x version of the LEAF board can then be
specified in the topology file CFG section to select the right
combination of optional boards in the system. An example will
be:
MyIbSystem N001 CFG: leaf2=12x
In this case leaf1 will be 4x as no special modifier is
defined for it (and LEAF is by default a 4x leaf). Leaf2
will be 12x as defined in the CFG section.
e) Special considerations should be made to design the top
system section when several optional board types exist. Top
system section might include multiple definitions for front
panel plugs like P1-4x and P1-12x (yes these are just names
that should follow the writing on the front or back
panels). As the different flavors of the boards are not
required to define the same names for their ports including
a 12x port might leave some of the top level connections
dangling (not connected) and thus the front panel ports of
the other flavor will be removed from the final system
definition. As an example let us consider a simple board
LEAF with 3 4x port flavor and one 12x port flavors. We
recommend connecting it in the top level using the
following scheme:
SYSTEM LEAF:4x
NODE U1
1 -4x-> 4XP1
2 -4x-> 4XP2
3 -4x-> 4XP3
...
SYSTEM LEAF:12x
NODE U1
1 -12x-> 12XP1
...
TOPSYSTEM myIbSystem
SUBSYSTEM LEAF leaf1
4XP1 -> L1/P1
4XP2 -> L1/P2
4XP3 -> L1/P3
12XP1 -> L1/P1-12x
f) Place the file in the <prefix>/lib/IBDM/ibdm1.0/ibnl
directory
g) Check the new file syntax by placing it in the ibnl
directory as described above, creating a simple
topology (ibdm-topo-file file with one such system and
running ibdmtr utility to parse it tracing a simple
path through it.
4. Formal definition in YACC Syntax:
INT ::= ([1-9][0-9]*0) ;
WIDTH ::= (4x8x12x) ;
SPEED ::= (2.5G5G10G) ;
NODETYPE ::= (SWCAHCA) ;
NAME ::= ([\[\]\\\*/A-Za-z0-9_.:%@~]+) ;
NL: LINE NL LINE;
ONL: NL;
ibnl: ONL systems topsystem;
systems: systems system ;
sub_inst_attributes: sub_inst_attributes sub_inst_attribute NL;
sub_inst_attribute: NAME '=' NAME '=' NAME NAME '=' NAME '=' INT
NAME '=' NAME ;
topsystem: TOPSYSTEM system_names NL sub_inst_attributes insts ;
system: SYSTEM system_names NL insts ;
system_names: system_name system_names ',' system_name ;
system_name: NAME ;
insts: insts node insts subsystem ;
node: node_header NL node_connections ;
node_header: NODE NODETYPE INT NAME NAME ;
node_connections: node_connections node_connection NL ;
node_connection: node_to_node_link node_to_port_link ;
node_to_node_link: INT '-' WIDTH '-' SPEED '-' '>' NAME INT '-' WIDTH
'-' '>' NAME INT INT '-' SPEED '-' '>' NAME INT INT '-' '>' NAME
INT ;
node_to_port_link: INT '-' WIDTH '-' SPEED '-' '>' NAME INT '-' WIDTH
'-' '>' NAME INT '-' SPEED '-' '>' NAME INT '-' '>' NAME ;
subsystem: subsystem_header NL subsystem_connections ;
subsystem_header: SUBSYSTEM NAME NAME ;
subsystem_connections: subsystem_connections subsystem_connection NL
;
subsystem_connection: subsystem_to_subsystem_link
subsystem_to_port_link ;
subsystem_to_subsystem_link: NAME '-' WIDTH '-' SPEED '-' '>' NAME NAME
NAME '-' WIDTH '-' '>' NAME NAME NAME '-' SPEED '-' '>' NAME NAME
NAME '-' '>' NAME NAME ;
subsystem_to_port_link: NAME '-' WIDTH '-' SPEED '-' '>' NAME NAME
'-' WIDTH '-' '>' NAME NAME '-' SPEED '-' '>' NAME NAME '-' '>'
NAME ;
AUTHOR
Eitan Zahavi, Mellanox Technologies LTD, eitan@mellanox.co.il
Opportunity
Personal Opportunity - Free software gives you access to billions of dollars of software at no cost. Use this software for your business, personal use or to develop a profitable skill. Access to source code provides access to a level of capabilities/information that companies protect though copyrights. Open source is a core component of the Internet and it is available to you. Leverage the billions of dollars in resources and capabilities to build a career, establish a business or change the world. The potential is endless for those who understand the opportunity.
Business Opportunity - Goldman Sachs, IBM and countless large corporations are leveraging open source to reduce costs, develop products and increase their bottom lines. Learn what these companies know about open source and how open source can give you the advantage.
Free Software
Free Software provides computer programs and capabilities at no cost but more importantly, it provides the freedom to run, edit, contribute to, and share the software. The importance of free software is a matter of access, not price. Software at no cost is a benefit but ownership rights to the software and source code is far more significant.
Free Office Software - The Libre Office suite provides top desktop productivity tools for free. This includes, a word processor, spreadsheet, presentation engine, drawing and flowcharting, database and math applications. Libre Office is available for Linux or Windows.
Free Books
The Free Books Library is a collection of thousands of the most popular public domain books in an online readable format. The collection includes great classical literature and more recent works where the U.S. copyright has expired. These books are yours to read and use without restrictions.
Source Code - Want to change a program or know how it works? Open Source provides the source code for its programs so that anyone can use, modify or learn how to write those programs themselves. Visit the GNU source code repositories to download the source.
Education
Study at Harvard, Stanford or MIT - Open edX provides free online courses from Harvard, MIT, Columbia, UC Berkeley and other top Universities. Hundreds of courses for almost all major subjects and course levels. Open edx also offers some paid courses and selected certifications.
Linux Manual Pages - A man or manual page is a form of software documentation found on Linux/Unix operating systems. Topics covered include computer programs (including library and system calls), formal standards and conventions, and even abstract concepts.
