요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
============
Introduction
============
The RapidIO standard is a packet-based fabric interconnect standard designed for
use in embedded systems. Development of the RapidIO standard is directed by the
RapidIO Trade Association (RTA). The current version of the RapidIO specification
is publicly available for download from the RTA web-site [1].
This document describes the basics of the Linux RapidIO subsystem and provides
information on its major components.
1 Overview
==========
Because the RapidIO subsystem follows the Linux device model it is integrated
into the kernel similarly to other buses by defining RapidIO-specific device and
bus types and registering them within the device model.
The Linux RapidIO subsystem is architecture independent and therefore defines
architecture-specific interfaces that provide support for common RapidIO
subsystem operations.
2. Core Components
==================
A typical RapidIO network is a combination of endpoints and switches.
Each of these components is represented in the subsystem by an associated data
structure. The core logical components of the RapidIO subsystem are defined
in include/linux/rio.h file.
2.1 Master Port
---------------
A master port (or mport) is a RapidIO interface controller that is local to the
processor executing the Linux code. A master port generates and receives RapidIO
packets (transactions). In the RapidIO subsystem each master port is represented
by a rio_mport data structure. This structure contains master port specific
resources such as mailboxes and doorbells. The rio_mport also includes a unique
host device ID that is valid when a master port is configured as an enumerating
host.
RapidIO master ports are serviced by subsystem specific mport device drivers
that provide functionality defined for this subsystem. To provide a hardware
independent interface for RapidIO subsystem operations, rio_mport structure
includes rio_ops data structure which contains pointers to hardware specific
implementations of RapidIO functions.
2.2 Device
----------
A RapidIO device is any endpoint (other than mport) or switch in the network.
All devices are presented in the RapidIO subsystem by corresponding rio_dev data
structure. Devices form one global device list and per-network device lists
(depending on number of available mports and networks).
2.3 Switch
----------
A RapidIO switch is a special class of device that routes packets between its
ports towards their final destination. The packet destination port within a
switch is defined by an internal routing table. A switch is presented in the
RapidIO subsystem by rio_dev data structure expanded by additional rio_switch
data structure, which contains switch specific information such as copy of the
routing table and pointers to switch specific functions.
The RapidIO subsystem defines the format and initialization method for subsystem
specific switch drivers that are designed to provide hardware-specific
implementation of common switch management routines.
2.4 Network
-----------
A RapidIO network is a combination of interconnected endpoint and switch devices.
Each RapidIO network known to the system is represented by corresponding rio_net
data structure. This structure includes lists of all devices and local master
ports that form the same network. It also contains a pointer to the default
master port that is used to communicate with devices within the network.
2.5 Device Drivers
------------------
RapidIO device-specific drivers follow Linux Kernel Driver Model and are
intended to support specific RapidIO devices attached to the RapidIO network.
2.6 Subsystem Interfaces
------------------------
RapidIO interconnect specification defines features that may be used to provide
one or more common service layers for all participating RapidIO devices. These
common services may act separately from device-specific drivers or be used by
device-specific drivers. Example of such service provider is the RIONET driver
which implements Ethernet-over-RapidIO interface. Because only one driver can be
registered for a device, all common RapidIO services have to be registered as
subsystem interfaces. This allows to have multiple common services attached to
the same device without blocking attachment of a device-specific driver.
3. Subsystem Initialization
===========================
In order to initialize the RapidIO subsystem, a platform must initialize and
register at least one master port within the RapidIO network. To register mport
within the subsystem controller driver's initialization code calls function
rio_register_mport() for each available master port.
After all active master ports are registered with a RapidIO subsystem,
an enumeration and/or discovery routine may be called automatically or
by user-space command.
RapidIO subsystem can be configured to be built as a statically linked or
modular component of the kernel (see details below).
4. Enumeration and Discovery
============================
4.1 Overview
------------
RapidIO subsystem configuration options allow users to build enumeration and
discovery methods as statically linked components or loadable modules.
An enumeration/discovery method implementation and available input parameters
define how any given method can be attached to available RapidIO mports:
simply to all available mports OR individually to the specified mport device.
Depending on selected enumeration/discovery build configuration, there are
several methods to initiate an enumeration and/or discovery process:
(a) Statically linked enumeration and discovery process can be started
automatically during kernel initialization time using corresponding module
parameters. This was the original method used since introduction of RapidIO
subsystem. Now this method relies on enumerator module parameter which is
'rio-scan.scan' for existing basic enumeration/discovery method.
When automatic start of enumeration/discovery is used a user has to ensure
that all discovering endpoints are started before the enumerating endpoint
and are waiting for enumeration to be completed.
Configuration option CONFIG_RAPIDIO_DISC_TIMEOUT defines time that discovering
endpoint waits for enumeration to be completed. If the specified timeout
expires the discovery process is terminated without obtaining RapidIO network
information. NOTE: a timed out discovery process may be restarted later using
a user-space command as it is described below (if the given endpoint was
enumerated successfully).
(b) Statically linked enumeration and discovery process can be started by
a command from user space. This initiation method provides more flexibility
for a system startup compared to the option (a) above. After all participating
endpoints have been successfully booted, an enumeration process shall be
started first by issuing a user-space command, after an enumeration is
completed a discovery process can be started on all remaining endpoints.
(c) Modular enumeration and discovery process can be started by a command from
user space. After an enumeration/discovery module is loaded, a network scan
process can be started by issuing a user-space command.
Similar to the option (b) above, an enumerator has to be started first.
(d) Modular enumeration and discovery process can be started by a module
initialization routine. In this case an enumerating module shall be loaded
first.
When a network scan process is started it calls an enumeration or discovery
routine depending on the configured role of a master port: host or agent.
Enumeration is performed by a master port if it is configured as a host port by
assigning a host destination ID greater than or equal to zero. The host
destination ID can be assigned to a master port using various methods depending
on RapidIO subsystem build configuration:
(a) For a statically linked RapidIO subsystem core use command line parameter
"rapidio.hdid=" with a list of destination ID assignments in order of mport
device registration. For example, in a system with two RapidIO controllers
the command line parameter "rapidio.hdid=-1,7" will result in assignment of
the host destination ID=7 to the second RapidIO controller, while the first
one will be assigned destination ID=-1.
(b) If the RapidIO subsystem core is built as a loadable module, in addition
to the method shown above, the host destination ID(s) can be specified using
traditional methods of passing module parameter "hdid=" during its loading:
- from command line: "modprobe rapidio hdid=-1,7", or
- from modprobe configuration file using configuration command "options",
like in this example: "options rapidio hdid=-1,7". An example of modprobe
configuration file is provided in the section below.
NOTES:
(i) if "hdid=" parameter is omitted all available mport will be assigned
destination ID = -1;
(ii) the "hdid=" parameter in systems with multiple mports can have
destination ID assignments omitted from the end of list (default = -1).
If the host device ID for a specific master port is set to -1, the discovery
process will be performed for it.
The enumeration and discovery routines use RapidIO maintenance transactions
to access the configuration space of devices.
NOTE: If RapidIO switch-specific device drivers are built as loadable modules
they must be loaded before enumeration/discovery process starts.
This requirement is cased by the fact that enumeration/discovery methods invoke
vendor-specific callbacks on early stages.
4.2 Automatic Start of Enumeration and Discovery
------------------------------------------------
Automatic enumeration/discovery start method is applicable only to built-in
enumeration/discovery RapidIO configuration selection. To enable automatic
enumeration/discovery start by existing basic enumerator method set use boot
command line parameter "rio-scan.scan=1".
This configuration requires synchronized start of all RapidIO endpoints that
form a network which will be enumerated/discovered. Discovering endpoints have
to be started before an enumeration starts to ensure that all RapidIO
controllers have been initialized and are ready to be discovered. Configuration
parameter CONFIG_RAPIDIO_DISC_TIMEOUT defines time (in seconds) which
a discovering endpoint will wait for enumeration to be completed.
When automatic enumeration/discovery start is selected, basic method's
initialization routine calls rio_init_mports() to perform enumeration or
discovery for all known mport devices.
Depending on RapidIO network size and configuration this automatic
enumeration/discovery start method may be difficult to use due to the
requirement for synchronized start of all endpoints.
4.3 User-space Start of Enumeration and Discovery
-------------------------------------------------
User-space start of enumeration and discovery can be used with built-in and
modular build configurations. For user-space controlled start RapidIO subsystem
creates the sysfs write-only attribute file '/sys/bus/rapidio/scan'. To initiate
an enumeration or discovery process on specific mport device, a user needs to
write mport_ID (not RapidIO destination ID) into that file. The mport_ID is a
sequential number (0 ... RIO_MAX_MPORTS) assigned during mport device
registration. For example for machine with single RapidIO controller, mport_ID
for that controller always will be 0.
To initiate RapidIO enumeration/discovery on all available mports a user may
write '-1' (or RIO_MPORT_ANY) into the scan attribute file.
4.4 Basic Enumeration Method
----------------------------
This is an original enumeration/discovery method which is available since
first release of RapidIO subsystem code. The enumeration process is
implemented according to the enumeration algorithm outlined in the RapidIO
Interconnect Specification: Annex I [1].
This method can be configured as statically linked or loadable module.
The method's single parameter "scan" allows to trigger the enumeration/discovery
process from module initialization routine.
This enumeration/discovery method can be started only once and does not support
unloading if it is built as a module.
The enumeration process traverses the network using a recursive depth-first
algorithm. When a new device is found, the enumerator takes ownership of that
device by writing into the Host Device ID Lock CSR. It does this to ensure that
the enumerator has exclusive right to enumerate the device. If device ownership
is successfully acquired, the enumerator allocates a new rio_dev structure and
initializes it according to device capabilities.
If the device is an endpoint, a unique device ID is assigned to it and its value
is written into the device's Base Device ID CSR.
If the device is a switch, the enumerator allocates an additional rio_switch
structure to store switch specific information. Then the switch's vendor ID and
device ID are queried against a table of known RapidIO switches. Each switch
table entry contains a pointer to a switch-specific initialization routine that
initializes pointers to the rest of switch specific operations, and performs
hardware initialization if necessary. A RapidIO switch does not have a unique
device ID; it relies on hopcount and routing for device ID of an attached
endpoint if access to its configuration registers is required. If a switch (or
chain of switches) does not have any endpoint (except enumerator) attached to
it, a fake device ID will be assigned to configure a route to that switch.
In the case of a chain of switches without endpoint, one fake device ID is used
to configure a route through the entire chain and switches are differentiated by
their hopcount value.
For both endpoints and switches the enumerator writes a unique component tag
into device's Component Tag CSR. That unique value is used by the error
management notification mechanism to identify a device that is reporting an
error management event.
Enumeration beyond a switch is completed by iterating over each active egress
port of that switch. For each active link, a route to a default device ID
(0xFF for 8-bit systems and 0xFFFF for 16-bit systems) is temporarily written
into the routing table. The algorithm recurs by calling itself with hopcount + 1
and the default device ID in order to access the device on the active port.
After the host has completed enumeration of the entire network it releases
devices by clearing device ID locks (calls rio_clear_locks()). For each endpoint
in the system, it sets the Discovered bit in the Port General Control CSR
to indicate that enumeration is completed and agents are allowed to execute
passive discovery of the network.
The discovery process is performed by agents and is similar to the enumeration
process that is described above. However, the discovery process is performed
without changes to the existing routing because agents only gather information
about RapidIO network structure and are building an internal map of discovered
devices. This way each Linux-based component of the RapidIO subsystem has
a complete view of the network. The discovery process can be performed
simultaneously by several agents. After initializing its RapidIO master port
each agent waits for enumeration completion by the host for the configured wait
time period. If this wait time period expires before enumeration is completed,
an agent skips RapidIO discovery and continues with remaining kernel
initialization.
4.5 Adding New Enumeration/Discovery Method
-------------------------------------------
RapidIO subsystem code organization allows addition of new enumeration/discovery
methods as new configuration options without significant impact to the core
RapidIO code.
A new enumeration/discovery method has to be attached to one or more mport
devices before an enumeration/discovery process can be started. Normally,
method's module initialization routine calls rio_register_scan() to attach
an enumerator to a specified mport device (or devices). The basic enumerator
implementation demonstrates this process.
4.6 Using Loadable RapidIO Switch Drivers
-----------------------------------------
In the case when RapidIO switch drivers are built as loadable modules a user
must ensure that they are loaded before the enumeration/discovery starts.
This process can be automated by specifying pre- or post- dependencies in the
RapidIO-specific modprobe configuration file as shown in the example below.
File /etc/modprobe.d/rapidio.conf::
# Configure RapidIO subsystem modules
# Set enumerator host destination ID (overrides kernel command line option)
options rapidio hdid=-1,2
# Load RapidIO switch drivers immediately after rapidio core module was loaded
softdep rapidio post: idt_gen2 idtcps tsi57x
# OR :
# Load RapidIO switch drivers just before rio-scan enumerator module is loaded
softdep rio-scan pre: idt_gen2 idtcps tsi57x
--------------------------
NOTE:
In the example above, one of "softdep" commands must be removed or
commented out to keep required module loading sequence.
5. References
=============
[1] RapidIO Trade Association. RapidIO Interconnect Specifications.
http://www.rapidio.org.
[2] Rapidio TA. Technology Comparisons.
http://www.rapidio.org/education/technology_comparisons/
[3] RapidIO support for Linux.
https://lwn.net/Articles/139118/
[4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005
https://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
RapidIO standard와 문서 범위
1-12RapidIO standard는 embedded system에서 사용하도록 설계된 packet 기반 fabric interconnect standard입니다. RapidIO Trade Association(RTA)이 standard 개발을 주도하며 현재 RapidIO specification은 RTA web site의 reference [1]에서 공개적으로 내려받을 수 있습니다.
이 문서는 Linux RapidIO subsystem의 기초와 주요 component를 설명합니다.
============
Introduction
============
The RapidIO standard is a packet-based fabric interconnect standard designed for
use in embedded systems. Development of the RapidIO standard is directed by the
RapidIO Trade Association (RTA). The current version of the RapidIO specification
is publicly available for download from the RTA web-site [1].
This document describes the basics of the Linux RapidIO subsystem and provides
information on its major components.
Linux device model 통합
13-23RapidIO subsystem은 Linux device model을 따릅니다. 다른 bus와 마찬가지로 RapidIO 전용 device type과 bus type을 정의하고 device model에 등록해 kernel에 통합합니다.
Linux RapidIO subsystem 자체는 architecture independent합니다. 공통 RapidIO subsystem operation을 지원하기 위해 architecture-specific interface를 별도로 정의합니다.
공통 core는 architecture independent하고 platform 구현은 architecture-specific interface를 제공합니다.
1 Overview
==========
Because the RapidIO subsystem follows the Linux device model it is integrated
into the kernel similarly to other buses by defining RapidIO-specific device and
bus types and registering them within the device model.
The Linux RapidIO subsystem is architecture independent and therefore defines
architecture-specific interfaces that provide support for common RapidIO
subsystem operations.
Core component와 `include/linux/rio.h`
24-31일반적인 RapidIO network는 endpoint와 switch의 조합입니다. Subsystem은 각 component를 대응하는 data structure로 표현합니다.
RapidIO subsystem의 core logical component는 `include/linux/rio.h`에 정의되어 있습니다.
2. Core Components
==================
A typical RapidIO network is a combination of endpoints and switches.
Each of these components is represented in the subsystem by an associated data
structure. The core logical components of the RapidIO subsystem are defined
in include/linux/rio.h file.
Master port와 `rio_ops`
32-48Master port 또는 mport는 Linux code를 실행하는 processor에 local인 RapidIO interface controller입니다. RapidIO packet, 즉 transaction을 생성하고 수신합니다.
Subsystem에서 각 master port는 `rio_mport` data structure로 표현됩니다. 이 structure는 mailbox와 doorbell 같은 mport-specific resource를 담고, mport가 enumerating host로 설정되었을 때 유효한 unique host device ID도 포함합니다.
RapidIO 전용 mport device driver가 master port를 서비스합니다. Hardware-independent subsystem interface를 제공하기 위해 `rio_mport` 안의 `rio_ops` structure가 hardware-specific RapidIO function 구현 pointer를 보관합니다.
Local controller의 resource와 operation table을 `rio_mport`가 묶습니다.
2.1 Master Port
---------------
A master port (or mport) is a RapidIO interface controller that is local to the
processor executing the Linux code. A master port generates and receives RapidIO
packets (transactions). In the RapidIO subsystem each master port is represented
by a rio_mport data structure. This structure contains master port specific
resources such as mailboxes and doorbells. The rio_mport also includes a unique
host device ID that is valid when a master port is configured as an enumerating
host.
RapidIO master ports are serviced by subsystem specific mport device drivers
that provide functionality defined for this subsystem. To provide a hardware
independent interface for RapidIO subsystem operations, rio_mport structure
includes rio_ops data structure which contains pointers to hardware specific
implementations of RapidIO functions.
RapidIO device와 device list
49-56RapidIO device는 mport를 제외한 network endpoint 또는 switch입니다. 모든 device는 대응하는 `rio_dev` data structure로 subsystem에 나타납니다.
Device는 하나의 global device list와 network별 device list를 구성합니다. Network별 list 수는 사용 가능한 mport와 network 수에 따라 달라집니다.
2.2 Device
----------
A RapidIO device is any endpoint (other than mport) or switch in the network.
All devices are presented in the RapidIO subsystem by corresponding rio_dev data
structure. Devices form one global device list and per-network device lists
(depending on number of available mports and networks).
Switch routing과 `rio_switch`
57-70RapidIO switch는 packet을 port 사이에서 최종 destination 방향으로 routing하는 특수 device입니다. Switch 내부 routing table이 packet의 destination port를 결정합니다.
Subsystem은 switch를 `rio_dev`로 표현하고 `rio_switch` data structure를 추가해 확장합니다. `rio_switch`에는 routing table 사본과 switch-specific function pointer 같은 정보가 들어 있습니다.
RapidIO subsystem은 공통 switch management routine의 hardware-specific 구현을 제공하는 switch driver의 형식과 초기화 방법을 정의합니다.
Generic device 위에 switch-specific state와 operation을 확장합니다.
2.3 Switch
----------
A RapidIO switch is a special class of device that routes packets between its
ports towards their final destination. The packet destination port within a
switch is defined by an internal routing table. A switch is presented in the
RapidIO subsystem by rio_dev data structure expanded by additional rio_switch
data structure, which contains switch specific information such as copy of the
routing table and pointers to switch specific functions.
The RapidIO subsystem defines the format and initialization method for subsystem
specific switch drivers that are designed to provide hardware-specific
implementation of common switch management routines.
Network, device driver, subsystem interface
71-97RapidIO network는 서로 연결된 endpoint와 switch device의 조합이며 `rio_net` data structure로 표현됩니다. 이 structure는 같은 network를 구성하는 모든 device와 local master port list를 담고, network 내 device와 통신할 때 사용할 default master port pointer도 가집니다.
RapidIO device-specific driver는 Linux Kernel Driver Model을 따르며 network에 연결된 특정 RapidIO device를 지원합니다.
RapidIO specification에는 모든 참여 device에 공통 service layer를 제공할 수 있는 기능이 있습니다. 이러한 service는 device-specific driver와 독립적으로 동작하거나 그 driver가 사용할 수 있습니다. 예로 RIONET driver는 Ethernet-over-RapidIO interface를 구현합니다.
한 device에는 driver 하나만 등록할 수 있으므로 공통 RapidIO service는 subsystem interface로 등록해야 합니다. 그러면 device-specific driver attachment를 막지 않으면서 같은 device에 여러 공통 service를 붙일 수 있습니다.
2.4 Network
-----------
A RapidIO network is a combination of interconnected endpoint and switch devices.
Each RapidIO network known to the system is represented by corresponding rio_net
data structure. This structure includes lists of all devices and local master
ports that form the same network. It also contains a pointer to the default
master port that is used to communicate with devices within the network.
2.5 Device Drivers
------------------
RapidIO device-specific drivers follow Linux Kernel Driver Model and are
intended to support specific RapidIO devices attached to the RapidIO network.
2.6 Subsystem Interfaces
------------------------
RapidIO interconnect specification defines features that may be used to provide
one or more common service layers for all participating RapidIO devices. These
common services may act separately from device-specific drivers or be used by
device-specific drivers. Example of such service provider is the RIONET driver
which implements Ethernet-over-RapidIO interface. Because only one driver can be
registered for a device, all common RapidIO services have to be registered as
subsystem interfaces. This allows to have multiple common services attached to
the same device without blocking attachment of a device-specific driver.
Subsystem 초기화와 mport 등록
98-112Platform이 RapidIO subsystem을 초기화하려면 RapidIO network 안의 master port를 최소 하나 초기화하고 등록해야 합니다. Controller driver initialization code는 사용 가능한 각 mport에 대해 `rio_register_mport()`를 호출합니다.
모든 active master port를 subsystem에 등록한 뒤 enumeration 또는 discovery routine을 자동으로 호출하거나 userspace command로 시작할 수 있습니다.
RapidIO subsystem은 kernel에 statically linked component 또는 modular component로 build할 수 있습니다.
Platform mport 등록이 끝난 후 network enumeration 또는 discovery를 시작합니다.
3. Subsystem Initialization
===========================
In order to initialize the RapidIO subsystem, a platform must initialize and
register at least one master port within the RapidIO network. To register mport
within the subsystem controller driver's initialization code calls function
rio_register_mport() for each available master port.
After all active master ports are registered with a RapidIO subsystem,
an enumeration and/or discovery routine may be called automatically or
by user-space command.
RapidIO subsystem can be configured to be built as a statically linked or
modular component of the kernel (see details below).
Enumeration·discovery 시작 방식
113-158Configuration option에 따라 enumeration과 discovery method는 static component 또는 loadable module로 build할 수 있습니다. Method 구현과 input parameter는 모든 mport에 붙일지 특정 mport에만 붙일지를 결정합니다.
방식 (a)는 statically linked method를 kernel initialization 중 module parameter로 자동 시작합니다. Basic method는 `rio-scan.scan` parameter를 사용합니다. 모든 discovering endpoint가 enumerating endpoint보다 먼저 시작되어 완료를 기다려야 합니다.
`CONFIG_RAPIDIO_DISC_TIMEOUT`은 discovering endpoint가 enumeration 완료를 기다리는 시간을 정의합니다. Timeout이면 network 정보를 얻지 못한 채 discovery를 종료하지만, endpoint가 성공적으로 enumerate되었다면 나중에 userspace command로 다시 시작할 수 있습니다.
방식 (b)는 statically linked method를 userspace command로 시작합니다. 모든 endpoint boot 후 enumeration을 먼저 시작하고 완료된 뒤 나머지 endpoint의 discovery를 시작하므로 자동 방식보다 startup을 유연하게 제어할 수 있습니다.
방식 (c)는 modular method를 load한 뒤 userspace command로 network scan을 시작합니다. 이 경우에도 enumerator가 먼저 시작되어야 합니다. 방식 (d)는 module initialization routine이 modular method를 시작하며 enumerating module을 먼저 load해야 합니다.
4. Enumeration and Discovery
============================
4.1 Overview
------------
RapidIO subsystem configuration options allow users to build enumeration and
discovery methods as statically linked components or loadable modules.
An enumeration/discovery method implementation and available input parameters
define how any given method can be attached to available RapidIO mports:
simply to all available mports OR individually to the specified mport device.
Depending on selected enumeration/discovery build configuration, there are
several methods to initiate an enumeration and/or discovery process:
(a) Statically linked enumeration and discovery process can be started
automatically during kernel initialization time using corresponding module
parameters. This was the original method used since introduction of RapidIO
subsystem. Now this method relies on enumerator module parameter which is
'rio-scan.scan' for existing basic enumeration/discovery method.
When automatic start of enumeration/discovery is used a user has to ensure
that all discovering endpoints are started before the enumerating endpoint
and are waiting for enumeration to be completed.
Configuration option CONFIG_RAPIDIO_DISC_TIMEOUT defines time that discovering
endpoint waits for enumeration to be completed. If the specified timeout
expires the discovery process is terminated without obtaining RapidIO network
information. NOTE: a timed out discovery process may be restarted later using
a user-space command as it is described below (if the given endpoint was
enumerated successfully).
(b) Statically linked enumeration and discovery process can be started by
a command from user space. This initiation method provides more flexibility
for a system startup compared to the option (a) above. After all participating
endpoints have been successfully booted, an enumeration process shall be
started first by issuing a user-space command, after an enumeration is
completed a discovery process can be started on all remaining endpoints.
(c) Modular enumeration and discovery process can be started by a command from
user space. After an enumeration/discovery module is loaded, a network scan
process can be started by issuing a user-space command.
Similar to the option (b) above, an enumerator has to be started first.
(d) Modular enumeration and discovery process can be started by a module
initialization routine. In this case an enumerating module shall be loaded
first.
Host·agent role과 `hdid` 지정
159-181Network scan은 master port의 role에 따라 enumeration routine 또는 discovery routine을 호출합니다. Host destination ID가 0 이상이면 해당 master port는 host port로 설정되어 enumeration을 수행합니다.
Statically linked subsystem core에서는 kernel command-line `rapidio.hdid=`에 mport 등록 순서대로 destination ID list를 지정합니다. 두 controller에 `rapidio.hdid=-1,7`을 주면 첫 controller는 -1, 두 번째는 host destination ID 7을 받습니다.
Subsystem core가 loadable module이면 위 kernel command line 외에도 module load 시 `hdid=`를 전달할 수 있습니다. 예시는 `modprobe rapidio hdid=-1,7`이며 modprobe configuration에는 `options rapidio hdid=-1,7`을 사용합니다.
Destination ID가 0 이상이면 host enumeration, -1이면 agent discovery 역할입니다.
When a network scan process is started it calls an enumeration or discovery
routine depending on the configured role of a master port: host or agent.
Enumeration is performed by a master port if it is configured as a host port by
assigning a host destination ID greater than or equal to zero. The host
destination ID can be assigned to a master port using various methods depending
on RapidIO subsystem build configuration:
(a) For a statically linked RapidIO subsystem core use command line parameter
"rapidio.hdid=" with a list of destination ID assignments in order of mport
device registration. For example, in a system with two RapidIO controllers
the command line parameter "rapidio.hdid=-1,7" will result in assignment of
the host destination ID=7 to the second RapidIO controller, while the first
one will be assigned destination ID=-1.
(b) If the RapidIO subsystem core is built as a loadable module, in addition
to the method shown above, the host destination ID(s) can be specified using
traditional methods of passing module parameter "hdid=" during its loading:
- from command line: "modprobe rapidio hdid=-1,7", or
- from modprobe configuration file using configuration command "options",
like in this example: "options rapidio hdid=-1,7". An example of modprobe
configuration file is provided in the section below.
`hdid` 기본값과 switch driver 선행 조건
182-200`hdid=` parameter를 생략하면 모든 mport의 destination ID는 -1입니다. Mport가 여러 개인 system에서는 list 끝의 assignment를 생략할 수 있고 생략된 값도 기본 -1입니다.
특정 master port의 host device ID가 -1이면 그 port는 discovery를 수행합니다. Enumeration과 discovery routine은 RapidIO maintenance transaction으로 device configuration space에 접근합니다.
RapidIO switch-specific driver를 loadable module로 build했다면 enumeration 또는 discovery 시작 전에 load해야 합니다. Enumeration/discovery method가 초기 단계부터 vendor-specific callback을 호출하기 때문입니다.
NOTES:
(i) if "hdid=" parameter is omitted all available mport will be assigned
destination ID = -1;
(ii) the "hdid=" parameter in systems with multiple mports can have
destination ID assignments omitted from the end of list (default = -1).
If the host device ID for a specific master port is set to -1, the discovery
process will be performed for it.
The enumeration and discovery routines use RapidIO maintenance transactions
to access the configuration space of devices.
NOTE: If RapidIO switch-specific device drivers are built as loadable modules
they must be loaded before enumeration/discovery process starts.
This requirement is cased by the fact that enumeration/discovery methods invoke
vendor-specific callbacks on early stages.
Automatic enumeration·discovery
201-223Automatic start는 built-in enumeration/discovery configuration에서만 사용할 수 있습니다. 기존 basic enumerator의 자동 시작은 boot command line에 `rio-scan.scan=1`을 설정합니다.
Network를 구성하는 모든 RapidIO endpoint의 시작을 동기화해야 합니다. 모든 controller가 초기화되어 discovery 가능한 상태가 되도록 discovering endpoint를 enumeration 시작 전에 먼저 기동해야 합니다. `CONFIG_RAPIDIO_DISC_TIMEOUT`은 완료를 기다릴 시간(초)을 지정합니다.
Automatic start를 선택하면 basic method initialization routine이 `rio_init_mports()`를 호출해 알려진 모든 mport에서 enumeration 또는 discovery를 수행합니다.
Network 크기와 구성에 따라 모든 endpoint의 synchronized start 요구사항 때문에 자동 방식은 사용하기 어려울 수 있습니다.
Discovering endpoint 준비와 controller 초기화가 모두 끝난 뒤 host scan을 시작해야 합니다.
4.2 Automatic Start of Enumeration and Discovery
------------------------------------------------
Automatic enumeration/discovery start method is applicable only to built-in
enumeration/discovery RapidIO configuration selection. To enable automatic
enumeration/discovery start by existing basic enumerator method set use boot
command line parameter "rio-scan.scan=1".
This configuration requires synchronized start of all RapidIO endpoints that
form a network which will be enumerated/discovered. Discovering endpoints have
to be started before an enumeration starts to ensure that all RapidIO
controllers have been initialized and are ready to be discovered. Configuration
parameter CONFIG_RAPIDIO_DISC_TIMEOUT defines time (in seconds) which
a discovering endpoint will wait for enumeration to be completed.
When automatic enumeration/discovery start is selected, basic method's
initialization routine calls rio_init_mports() to perform enumeration or
discovery for all known mport devices.
Depending on RapidIO network size and configuration this automatic
enumeration/discovery start method may be difficult to use due to the
requirement for synchronized start of all endpoints.
Userspace scan attribute
224-238Userspace start는 built-in과 modular configuration 모두에서 사용할 수 있습니다. RapidIO subsystem은 write-only sysfs attribute `/sys/bus/rapidio/scan`을 만듭니다.
특정 mport에서 enumeration 또는 discovery를 시작하려면 RapidIO destination ID가 아니라 `mport_ID`를 이 file에 씁니다. `mport_ID`는 mport 등록 중 0부터 `RIO_MAX_MPORTS` 범위에서 순차적으로 배정되며 controller 하나인 system에서는 항상 0입니다.
사용 가능한 모든 mport에서 scan하려면 `-1`, 즉 `RIO_MPORT_ANY`를 씁니다.
4.3 User-space Start of Enumeration and Discovery
-------------------------------------------------
User-space start of enumeration and discovery can be used with built-in and
modular build configurations. For user-space controlled start RapidIO subsystem
creates the sysfs write-only attribute file '/sys/bus/rapidio/scan'. To initiate
an enumeration or discovery process on specific mport device, a user needs to
write mport_ID (not RapidIO destination ID) into that file. The mport_ID is a
sequential number (0 ... RIO_MAX_MPORTS) assigned during mport device
registration. For example for machine with single RapidIO controller, mport_ID
for that controller always will be 0.
To initiate RapidIO enumeration/discovery on all available mports a user may
write '-1' (or RIO_MPORT_ANY) into the scan attribute file.
Basic method와 device ownership
239-263Basic enumeration/discovery method는 RapidIO subsystem 최초 release부터 제공된 original method이며 RapidIO Interconnect Specification Annex I [1]의 algorithm을 구현합니다.
Static 또는 loadable module로 구성할 수 있고 단일 `scan` parameter가 module initialization routine에서 process를 시작합니다. 이 method는 한 번만 시작할 수 있으며 module build에서도 unloading을 지원하지 않습니다.
Enumerator는 recursive depth-first algorithm으로 network를 순회합니다. 새 device를 찾으면 Host Device ID Lock CSR에 써서 ownership을 얻고 exclusive enumeration 권한을 확보합니다.
Ownership 획득에 성공하면 새 `rio_dev`를 할당해 capability에 따라 초기화합니다. Endpoint에는 unique device ID를 배정하고 Base Device ID CSR에 기록합니다.
Lock으로 device ownership을 확보한 뒤 자료 구조와 endpoint ID를 초기화합니다.
4.4 Basic Enumeration Method
----------------------------
This is an original enumeration/discovery method which is available since
first release of RapidIO subsystem code. The enumeration process is
implemented according to the enumeration algorithm outlined in the RapidIO
Interconnect Specification: Annex I [1].
This method can be configured as statically linked or loadable module.
The method's single parameter "scan" allows to trigger the enumeration/discovery
process from module initialization routine.
This enumeration/discovery method can be started only once and does not support
unloading if it is built as a module.
The enumeration process traverses the network using a recursive depth-first
algorithm. When a new device is found, the enumerator takes ownership of that
device by writing into the Host Device ID Lock CSR. It does this to ensure that
the enumerator has exclusive right to enumerate the device. If device ownership
is successfully acquired, the enumerator allocates a new rio_dev structure and
initializes it according to device capabilities.
If the device is an endpoint, a unique device ID is assigned to it and its value
is written into the device's Base Device ID CSR.
Switch initialization과 recursive routing
264-288Device가 switch이면 enumerator가 switch-specific 정보를 저장할 `rio_switch`를 추가로 할당합니다. Vendor ID와 device ID로 known RapidIO switch table을 조회하고, table entry의 initialization routine이 나머지 switch operation pointer와 필요한 hardware initialization을 설정합니다.
RapidIO switch 자체에는 unique device ID가 없습니다. Configuration register 접근에는 연결된 endpoint의 device ID를 위한 routing과 hopcount를 사용합니다. Enumerator 외 endpoint가 없는 switch 또는 switch chain에는 route 구성을 위한 fake device ID 하나를 배정하고 hopcount로 각 switch를 구분합니다.
Endpoint와 switch 모두에 unique component tag를 Component Tag CSR에 씁니다. Error-management notification은 이 값으로 event를 보고한 device를 식별합니다.
Switch 너머를 enumerate할 때 active egress port를 순회합니다. 각 active link의 routing table에 default device ID, 즉 8-bit system은 `0xFF`, 16-bit system은 `0xFFFF` route를 임시로 쓰고 `hopcount + 1`과 default ID로 algorithm을 재귀 호출합니다.
Switch 식별, route 준비, active egress별 재귀 순회가 이어집니다.
If the device is a switch, the enumerator allocates an additional rio_switch
structure to store switch specific information. Then the switch's vendor ID and
device ID are queried against a table of known RapidIO switches. Each switch
table entry contains a pointer to a switch-specific initialization routine that
initializes pointers to the rest of switch specific operations, and performs
hardware initialization if necessary. A RapidIO switch does not have a unique
device ID; it relies on hopcount and routing for device ID of an attached
endpoint if access to its configuration registers is required. If a switch (or
chain of switches) does not have any endpoint (except enumerator) attached to
it, a fake device ID will be assigned to configure a route to that switch.
In the case of a chain of switches without endpoint, one fake device ID is used
to configure a route through the entire chain and switches are differentiated by
their hopcount value.
For both endpoints and switches the enumerator writes a unique component tag
into device's Component Tag CSR. That unique value is used by the error
management notification mechanism to identify a device that is reporting an
error management event.
Enumeration beyond a switch is completed by iterating over each active egress
port of that switch. For each active link, a route to a default device ID
(0xFF for 8-bit systems and 0xFFFF for 16-bit systems) is temporarily written
into the routing table. The algorithm recurs by calling itself with hopcount + 1
and the default device ID in order to access the device on the active port.
Enumeration 완료와 agent discovery
289-306Host가 전체 network enumeration을 완료하면 `rio_clear_locks()`를 호출해 device ID lock을 지우고 device를 release합니다. 각 endpoint의 Port General Control CSR에서 Discovered bit를 설정해 완료를 알리고 agent가 passive discovery를 수행하도록 허용합니다.
Agent discovery는 enumeration과 유사하지만 기존 routing을 바꾸지 않습니다. Agent는 network 구조를 수집하고 discovered device의 internal map만 구성합니다. 따라서 각 Linux 기반 RapidIO component가 network 전체 view를 갖게 됩니다.
여러 agent가 discovery를 동시에 수행할 수 있습니다. 각 agent는 master port 초기화 후 설정된 wait time 동안 host enumeration 완료를 기다립니다. Timeout 전에 완료되지 않으면 discovery를 건너뛰고 나머지 kernel initialization을 계속합니다.
Host가 lock과 Discovered bit를 정리한 뒤 agent가 routing 변경 없이 topology를 수집합니다.
After the host has completed enumeration of the entire network it releases
devices by clearing device ID locks (calls rio_clear_locks()). For each endpoint
in the system, it sets the Discovered bit in the Port General Control CSR
to indicate that enumeration is completed and agents are allowed to execute
passive discovery of the network.
The discovery process is performed by agents and is similar to the enumeration
process that is described above. However, the discovery process is performed
without changes to the existing routing because agents only gather information
about RapidIO network structure and are building an internal map of discovered
devices. This way each Linux-based component of the RapidIO subsystem has
a complete view of the network. The discovery process can be performed
simultaneously by several agents. After initializing its RapidIO master port
each agent waits for enumeration completion by the host for the configured wait
time period. If this wait time period expires before enumeration is completed,
an agent skips RapidIO discovery and continues with remaining kernel
initialization.
새 enumeration·discovery method 추가
307-319RapidIO subsystem code organization은 core code에 큰 영향을 주지 않고 새 enumeration/discovery method를 configuration option으로 추가할 수 있게 설계되었습니다.
새 method는 process 시작 전에 하나 이상의 mport device에 attach해야 합니다. 일반적으로 method module initialization routine이 `rio_register_scan()`을 호출해 지정한 mport 또는 mport 집합에 enumerator를 붙입니다. Basic enumerator 구현이 이 절차의 예입니다.
Method module이 mport에 scanner를 붙인 뒤 enumeration 또는 discovery를 시작할 수 있습니다.
4.5 Adding New Enumeration/Discovery Method
-------------------------------------------
RapidIO subsystem code organization allows addition of new enumeration/discovery
methods as new configuration options without significant impact to the core
RapidIO code.
A new enumeration/discovery method has to be attached to one or more mport
devices before an enumeration/discovery process can be started. Normally,
method's module initialization routine calls rio_register_scan() to attach
an enumerator to a specified mport device (or devices). The basic enumerator
implementation demonstrates this process.
Loadable switch driver 순서
320-348RapidIO switch driver를 loadable module로 build한 경우 enumeration/discovery 전에 반드시 load해야 합니다. RapidIO 전용 modprobe configuration file의 pre-dependency 또는 post-dependency로 순서를 자동화할 수 있습니다.
예제 `/etc/modprobe.d/rapidio.conf`는 `options rapidio hdid=-1,2`로 kernel command-line 설정을 override합니다.
`softdep rapidio post: idt_gen2 idtcps tsi57x`는 rapidio core module 직후 switch driver를 load합니다. 대안인 `softdep rio-scan pre: idt_gen2 idtcps tsi57x`는 rio-scan enumerator module 직전에 load합니다.
두 `softdep` command 중 하나는 제거하거나 comment 처리해 필요한 module loading sequence 하나만 유지해야 합니다.
4.6 Using Loadable RapidIO Switch Drivers
-----------------------------------------
In the case when RapidIO switch drivers are built as loadable modules a user
must ensure that they are loaded before the enumeration/discovery starts.
This process can be automated by specifying pre- or post- dependencies in the
RapidIO-specific modprobe configuration file as shown in the example below.
File /etc/modprobe.d/rapidio.conf::
# Configure RapidIO subsystem modules
# Set enumerator host destination ID (overrides kernel command line option)
options rapidio hdid=-1,2
# Load RapidIO switch drivers immediately after rapidio core module was loaded
softdep rapidio post: idt_gen2 idtcps tsi57x
# OR :
# Load RapidIO switch drivers just before rio-scan enumerator module is loaded
softdep rio-scan pre: idt_gen2 idtcps tsi57x
--------------------------
NOTE:
In the example above, one of "softdep" commands must be removed or
commented out to keep required module loading sequence.
참고 자료
349-362참고 자료에는 RapidIO Trade Association의 Interconnect Specifications와 Technology Comparisons, LWN의 Linux RapidIO 지원 기사, Matt Porter의 Ottawa Linux Symposium 2005 논문이 포함됩니다.
5. References
=============
[1] RapidIO Trade Association. RapidIO Interconnect Specifications.
http://www.rapidio.org.
[2] Rapidio TA. Technology Comparisons.
http://www.rapidio.org/education/technology_comparisons/
[3] RapidIO support for Linux.
https://lwn.net/Articles/139118/
[4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005
https://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf
요약과 해설
rapidio.rst:1-362Linux RapidIO subsystem은 mport, device, switch, network와 subsystem interface를 Linux device model에 통합합니다. Host는 destination ID와 lock을 사용해 network를 depth-first enumerate하고 route를 구성하며, agent는 host 완료 후 routing을 바꾸지 않고 topology를 discover합니다. Built-in·module 및 kernel·userspace 시작 방식을 모두 지원합니다.