Reference Router (RR) Alpha Installation Instructions

These instructions assume that you have successfully installed a NetFPGA card with CentOS 4.4, as per the Hardware Setup Procedure, found at http://netfpga.org/alpha/ .

To synthesize a new bitfile from source, you will also need to have Xilinx ISE installed. Instructions can be found here: http://netfpga.org/alpha/Develop.html .

If any steps in the installation process fail, please email a detailed description of your issue to netfpga-alpha@lists.stanford.edu

Show Contents...Hide Contents...

Step 1. Connect Ethernet test cables

• Connect 'eth1' to 'nf2c0' (c0 is the port closest to the mainboard)
  
• Connect 'eth2' to 'nf2c1' (c1 is the port one away from the mainboard)
  
  • The location of your eth1 and eth2 ports may vary depending on your NIC
    
  • The photo below shows the configuration of a nf-test machine
    
    • 
      

Step 2. Log in as root

• Log in as root or 'su -' to root using an X session, because we will be testing the GUI Scone
  

Step 3. Download and unpack alpha release files

• download alpha release here
  

• Unpack the tar.gz file. We'll assume you unzipped the file in your home directory, ~/.
  

tar -zxvf NetFPGA_RR_1_0_Alpha.tar.gz

You are welcome to review the code and we welcome your feedback. We only ask that you not redistribute the code or post it on a public site because several universities are planning to teach router design courses and this code would provide students with the solutions to their assignments.

Step 4. Set up environment variables

• We assume that you will be using a bash terminal
  

• If this is the first NetFPGA release on this user account, append environment variables to ~/.bashrc, to be loaded upon login:
  

cat ~/NF2/bashrc_addon >> .bashrc

• If you've previously used your current account on the test computer with a NetFPGA release and have appended to .bashrc, appending a second time will produce a duplicate of some environment variables. You want to manually copy the variables as needed from bashrc_addon:
  

NF2_ROOT
NF2_DESIGN_DIR
NF2_WORK_DIR
PYTHONPATH
PERL5LIB
LD_LIBRARY_PATH

• Load environment variables for the current session:
  

source ~/.bashrc

Step 5. Verify kernel sources

• Make sure the machine is connected to the internet.
  

• Find the currently running kernel version, via `uname`.
  

uname -r

• Sample correct output:
  

2.6.9-42.ELsmp

• List kernel-devel sources. The yum list may take 20 seconds or more to create. This step verifies that you have the kernel sources necessary to build the driver, a kernel module.
  

yum list | grep kernel-smp-devel
:(no space between dash)

• Sample correct output:
  

kernel-smp-devel.i686 2.6.9-42.ELsmp installed
kernel-smp-devel.i586 2.6.9-55.0.9.ELsmp update
kernel-smp-devel.i686 2.6.9-55.0.9.ELsmp update

• If you can't find "installed" in the same line as the kernel-smp-devel package for your currently running kernel version, then you'll need to install kernel-smp-devel.i686 or similar.
  

Step 6. Compile driver and tools

• Compile
  

cd ~/NF2/
make

• Sample correct output:
  

make -C lib
make[1]: Entering directory `/home/grg/NF2/lib'
make -C C
make[2]: Entering directory `/home/grg/NF2/lib/C'
make -C kernel
make[3]: Entering directory `/home/grg/NF2/lib/C/kernel'
make -C /lib/modules/2.6.9-55.0.9.ELsmp/build M=/home/grg/NF2/lib/C/kernel LDDINC=/home/grg/NF2/lib/C/kernel/../include modules
make[4]: Entering directory `/usr/src/kernels/2.6.9-55.0.9.EL-smp-i686'
Building modules, stage 2.
MODPOST
make[4]: Leaving directory `/usr/src/kernels/2.6.9-55.0.9.EL-smp-i686'
make[3]: Leaving directory `/home/grg/NF2/lib/C/kernel'
make -C download
make[3]: Entering directory `/home/grg/NF2/lib/C/download'
make -C ../common
make[4]: Entering directory `/home/grg/NF2/lib/C/common'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home/grg/NF2/lib/C/common'
make[3]: Leaving directory `/home/grg/NF2/lib/C/download'
make -C reg_access
make[3]: Entering directory `/home/grg/NF2/lib/C/reg_access'
make -C ../common
make[4]: Entering directory `/home/grg/NF2/lib/C/common'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home/grg/NF2/lib/C/common'
make[3]: Leaving directory `/home/grg/NF2/lib/C/reg_access'
make -C router
make[3]: Entering directory `/home/grg/NF2/lib/C/router'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/home/grg/NF2/lib/C/router'
make[2]: Leaving directory `/home/grg/NF2/lib/C'
make -C scripts
make[2]: Entering directory `/home/grg/NF2/lib/scripts'
make -C cpci_reprogram
make[3]: Entering directory `/home/grg/NF2/lib/scripts/cpci_reprogram'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/home/grg/NF2/lib/scripts/cpci_reprogram'
make -C cpci_config_reg_access
make[3]: Entering directory `/home/grg/NF2/lib/scripts/cpci_config_reg_access'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/home/grg/NF2/lib/scripts/cpci_config_reg_access'
make[2]: Leaving directory `/home/grg/NF2/lib/scripts'
make[1]: Leaving directory `/home/grg/NF2/lib'
make -C bitfiles
make[1]: Entering directory `/home/grg/NF2/bitfiles'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/grg/NF2/bitfiles'
make -C projects/scone/base
make[1]: Entering directory `/home/grg/NF2/projects/scone/base'
gcc -g -Wall -D_DEBUG_ -D_LINUX_ -I lwtcp -D_GNU_SOURCE -D_CPUMODE_ -o scone sr_main.o -lnsl -lm -lresolv -lpthread -lpcap -lnet libsr_base.a liblwtcp.a
libsr_base.a /usr/lib/libnet.a
make[1]: Leaving directory `/home/grg/NF2/projects/scone/base'

• If you get an error message such as "make: * /lib/modules/2.6.9-42.ELsmp/build: No such file or directory. Stop.", then kernel sources are need to build the driver.
  

Step 7. Load driver and tools

• Install the driver and reboot. The driver will be stored in /lib/modules/`uname -r`/kernel/drivers/nf2.ko
  

make install

• Sample correct output:
  

for dir in lib bitfiles ; do
make -C $dir install;
done
make[1]: Entering directory `/home/gac1/temp_install/NF2/lib'
for dir in C scripts ; do
make -C $dir install;
done
make[2]: Entering directory `/home/gac1/temp_install/NF2/lib/C'
for dir in kernel download reg_access ; do
make -C $dir install;
done
make[3]: Entering directory `/home/gac1/temp_install/NF2/lib/C/kernel'
make -C /lib/modules/2.6.9-42.ELsmp/build M=/home/gac1/temp_install/NF2/lib/C/ke
rnel LDDINC=/home/gac1/temp_install/NF2/lib/C/kernel/../include modules
make[4]: Entering directory `/usr/src/kernels/2.6.9-42.EL-smp-i686'
Building modules, stage 2.
MODPOST
make[4]: Leaving directory `/usr/src/kernels/2.6.9-42.EL-smp-i686'
install -m 644 nf2.ko /lib/modules/`uname -r`/kernel/drivers/nf2.ko
/sbin/depmod -a
make[3]: Leaving directory `/home/gac1/temp_install/NF2/lib/C/kernel'
make[3]: Entering directory `/home/gac1/temp_install/NF2/lib/C/download'
install -C nf2_download /usr/local/bin
make[3]: Leaving directory `/home/gac1/temp_install/NF2/lib/C/download'
make[3]: Entering directory `/home/gac1/temp_install/NF2/lib/C/reg_access'
install -C regread /usr/local/bin
install -C regwrite /usr/local/bin
make[3]: Leaving directory `/home/gac1/temp_install/NF2/lib/C/reg_access'
make[2]: Leaving directory `/home/gac1/temp_install/NF2/lib/C'
make[2]: Entering directory `/home/gac1/temp_install/NF2/lib/scripts'
for dir in cpci_reprogram cpci_config_reg_access ; do
make -C $dir install;
done
make[3]: Entering directory `/home/gac1/temp_install/NF2/lib/scripts/cpci_reprog
ram'
install -C cpci_reprogram.pl /usr/local/sbin
make[3]: Leaving directory `/home/gac1/temp_install/NF2/lib/scripts/cpci_reprogr
am'
make[3]: Entering directory `/home/gac1/temp_install/NF2/lib/scripts/cpci_config
_reg_access'
install -C dumpregs.sh /usr/local/sbin
install -C loadregs.sh /usr/local/sbin
make[3]: Leaving directory `/home/gac1/temp_install/NF2/lib/scripts/cpci_config_
reg_access'
make[2]: Leaving directory `/home/gac1/temp_install/NF2/lib/scripts'
make[1]: Leaving directory `/home/gac1/temp_install/NF2/lib'
make[1]: Entering directory `/home/gac1/temp_install/NF2/bitfiles'
for bitfile in CPCI_2.1.bit cpci_reprogrammer.bit ; do
install -C -D -m 0644 $bitfile /usr/local/NF2/bitfiles/$bitfile ;
done
make[1]: Leaving directory `/home/gac1/temp_install/NF2/bitfiles'

• Reboot the machine. The Alpha 1.0 driver currently crashes upon rmmod, so a reboot is required to load the newly compiled driver. You may want to check if other users are on the machine with the 'who' command first. If you don't like the other current machine users or you're the only one on the machine, run the following:
  

reboot

• After reboot log in as root.
  

• Verify that the driver loaded:
  

lsmod | grep nf2

• Sample correct output:
  

nf2 29068 0

Step 8. Verify NetFPGA interfaces

• Verify that four nf2cX interfaces have successfully loaded:
  

ifconfig -a | grep nf2

• Sample correct output:
  

nf2c0 Link encap:Ethernet HWaddr 00:4E:46:32:43:00
nf2c1 Link encap:Ethernet HWaddr 00:4E:46:32:43:01
nf2c2 Link encap:Ethernet HWaddr 00:4E:46:32:43:02
nf2c3 Link encap:Ethernet HWaddr 00:4E:46:32:43:03

Step 9. Reprogram the CPCI

• Run the cpci reprogramming script
  

cpci_reprogram.pl --all
:(to reprogram all NetFPGAs in a system)

• Expected output:
  

Loading the CPCI Reprogrammer on NetFPGA 0
Found net device: nf2c0
Download successful
Loading the CPCI on NetFPGA 0
Found net device: nf2c0
Download successful
CPCI on NetFPGA 0 has been successfully reprogrammed

Step 10. Bring up interfaces

• Bring the nf2cX interfaces are brought up:
  

ifconfig nf2c0 up
ifconfig nf2c1 up
ifconfig nf2c2 up
ifconfig nf2c3 up

• Verify that four nf2cX interfaces have successfully loaded:
  

ifconfig

• Sample correct output:
  

nf2c0 Link encap:Ethernet HWaddr 00:4E:46:32:43:00
inet6 addr: fe80
24e:46ff:fe32:4300/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:34100 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2046000 (1.9 MiB) TX bytes:0 (0.0 b)
Interrupt:217

nf2c1 Link encap:Ethernet HWaddr 00:4E:46:32:43:01
inet addr:192.168.100.1 Bcast:192.168.100.255 Mask:255.
inet6 addr: fe80
24e:46ff:fe32:4301/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:34109 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2046540 (1.9 MiB) TX bytes:0 (0.0 b)
Interrupt:217

nf2c2 Link encap:Ethernet HWaddr 00:4E:46:32:43:02
inet6 addr: fe80
24e:46ff:fe32:4302/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:34100 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2046000 (1.9 MiB) TX bytes:0 (0.0 b)
Interrupt:217

nf2c3 Link encap:Ethernet HWaddr 00:4E:46:32:43:03
inet6 addr: fe80
24e:46ff:fe32:4303/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:34100 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2046000 (1.9 MiB) TX bytes:0 (0.0 b)
Interrupt:217

Step 11. Install the Perl Pcap libraries

• Install the perl Pcap library
  

yum -y install perl-Net-Pcap

• Sample correct output:
  

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
perl-Net-Pcap i386 0.12-1.el4.rf rpmforge 107 k
Installing for dependencies:
perl-IO-Interface i386 1.03-1.el4.rf rpmforge 52 k

Transaction Summary
=============================================================================
Install 2 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 159 k
Downloading Packages:
(1/2): perl-Net-Pcap-0.12 100% |=========================| 107 kB 00:00
(2/2): perl-IO-Interface- 100% |=========================| 52 kB 00:00
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: perl-IO-Interface ######################### [1/2]
Installing: perl-Net-Pcap ######################### [2/2]

Installed: perl-Net-Pcap.i386 0:0.12-1.el4.rf
Dependency Installed: perl-IO-Interface.i386 0:1.03-1.el4.rf
Complete!

Step 12. Install Perl Raw IP libraries

• Install the perl Raw IP library
  

yum -y install perl-Net-RawIP.i386

• Sample correct output:
  

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
perl-Net-RawIP i386 0.2-2.el4.rf rpmforge 106 k

Transaction Summary
=============================================================================
Install 1 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 106 k
Downloading Packages:
(1/1): perl-Net-RawIP-0.2 100% |=========================| 106 kB 00:00
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: perl-Net-RawIP ######################### [1/1]

Installed: perl-Net-RawIP.i386 0:0.2-2.el4.rf
Complete!

Step 13. Install Perl Error libraries

• Install the perl Error library
  

yum -y install perl-Error.noarch

• Sample correct output:
  

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
perl-Error noarch 0.17008-2.el4.rf rpmforge 26 k


Transaction Summary
=============================================================================
Install 1 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 26 k
Is this ok [y/N]: y
Downloading Packages:
(1/1): perl-Error-0.17008 100% |=========================| 26 kB 00:00
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: perl-Error ######################### [1/1]

Installed: perl-Error.noarch 0:0.17008-2.el4.rf
Complete!

Step 14. Install Java

• Download and install the Java JDK (JDK 6 Update 3) from SUN
  

Download from: http://java.sun.com/javase/downloads/index.jsp
Install instructions: http://java.sun.com/javase/6/webnotes/install/jre/install-linux.html

• Install the key for the JPackage repository
  

rpm --import http://jpackage.org/jpackage.asc

• Install the JPackage repository information for yum
  

cd /etc/yum.repos.d
wget http://www.jpackage.org/jpackage17.repo

• Install the Java JRE
  

yum -y --enablerepo=jpackage-generic-nonfree install java-1.6.0-sun-compat.i586

• Expected Output
  

Dependencies Resolved

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
java-1.6.0-sun-compat i586 1.6.0.03-1jpp jpackage-generic-nonfree
54 k

Transaction Summary
=============================================================================
Install 1 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 54 k
Is this ok [y/N]: y
Downloading Packages:
(1/1): java-1.6.0-sun-com 100% |=========================| 54 kB 00:00
Running Transaction Test
Finished Transaction Test

• Set default JAVA path to new JRE
  

Type: alternatives --config java
Expected Output
There are 2 programs which provide 'java'.

Selection Command
-----------------------------------------------
1 /usr/lib/jvm/jre-1.4.2-gcj/bin/java
*+ 2 /usr/lib/jvm/jre-1.6.0-sun/bin/java

Enter to keep the current selection[+], or type selection number:

Select number corresponding to jre-1.6.0-sun

Step 15. Load reference_router bitfile

• Download the reference bitfile to the NetFPGA board:
  

nf2_download ~/NF2/bitfiles/reference_router.bit

• Sample correct output:
  

Found net device: nf2c0
Bit file built from: nf2_top_par.ncd
Part: 2vp50ff1152
Date: 2007/10/ 9
Time: 22: 3: 4
Error Registers: 1000000
Good, after resetting programming interface the FIFO is empty
Download completed - 2377668 bytes. (expected 2377668).
DONE went high - chip has been successfully programmed.

Step 16. Run regression test suite

• Run the regression test suite. The tests should take about 10 minutes total.
  

~/NF2/bin/nf21_regress_test.pl

• Sample correct output:
  

Running tests on project 'driver'...
:Running test 'driver_compile'... PASS
:Running test 'driver_install'... PASS
:Running test 'verify_mtu'... PASS

Running tests on project 'reference_router'...
:Running global setup... PASS
:Running test 'test_router_cpusend/run.pl'... PASS
:Running test 'test_wrong_dest_mac'... PASS
:Running test 'test_nonip_packet'... PASS
:Running test 'test_nonipv4_packet'... PASS
:Running test 'test_invalidttl_packet'... PASS
:Running test 'test_lpm_misses'... PASS
:Running test 'test_arp_misses'... PASS
:Running test 'test_badipchecksum_packet'... PASS
:Running test 'test_ipdest_filter_hit'... PASS
:Running test 'test_packet_fowarding'... PASS
:Running test 'test_lpm'... PASS
:Running test 'test_lpm_next_hop'... PASS
:Running test 'test_queue_overflow'... PASS
:Running test 'test_oq_limit'... PASS
:Running test 'test_ipdest_filter'... PASS

Running tests on project 'scone'...
:Running global setup... PASS
:Running test 'test_build'... PASS
:Running test 'test_mac_set'... PASS
:Running test 'test_ip_set'... PASS
:Running test 'test_rtable_set'... PASS
:Running test 'test_disabled_interfaces/run.pl'... PASS
:Running test 'test_noniparp_ethtype'... PASS
:Running test 'test_arp_rpl'... PASS
:Running test 'test_arp_norpl'... PASS
:Running test 'test_arp_quepkt'... PASS
:Running test 'test_ip_error'... PASS
:Running test 'test_ip_checksum/run.pl'... PASS
:Running test 'test_ttl_expired/run.pl'... PASS
:Running test 'test_send_receive/run.pl'... PASS
:Running test 'test_arp_req/run.pl'... PASS
:Running test 'test_tcp_port/run.pl'... PASS
:Running test 'test_udp_packet/run.pl'... PASS
:Running test 'test_unknown_ip/run.pl'... PASS
:Running test 'test_icmp_echo/run.pl'... PASS
:Running test 'test_icmp_notecho/run.pl'... PASS
:Running global teardown... PASS

Running tests on project 'gui_scone'...
:Running global setup... PASS
:Running test 'test_main_frame'... PASS
:Running test 'test_routing_table'... PASS
:Running test 'test_arp_table'... PASS
:Running test 'test_port_config_table'... PASS
:Running global teardown... PASS

• If there are no errors (all tests say PASS), you can play with your router, or go on to creating a bitfile from source.
  

Step 17. Synthesize reference_router bitfile, from source

Note: This step will take about 45-60 mins.

• If you are a hardware developer and would like to synthesize your own NetFPGA Router hardware using the Verilog source code, follow the steps 17-19 listed below. To synthesize FPGA hardware, you will need to have all of the FPGA Development tools installed, as per http://NetFPGA.org/alpha/Develop.html
  

• Login, either direct in an X session or via ssh -X. This step causes ~/nf2_profile, plus environment variables, to be sourced. You may say "But I'm not running anything graphical!" and you'd be right. Unfortunately, even when called with no gui, the Xilinx tools require X to be running. A bugreport on this issue has been filed to Xilinx.
  

• Set up the Xilinx ISE tools (see Xilinx's website for instructions). Make sure the Xilnx tools are in your path and that the XILINX environment variable is set.
  

• Go to the synthesis directory for the reference_nic and run make. This step should take under an hour on a well-endowed machine.
  

cd ~/NF2/projects/reference_router/synth
time make

• Verify the reference_router bitfile (nf2_top_par.bit) has been created.
  

ls | grep nf2_top_par.bit

• Sample correct output:
  

nf2_top_par.bit

Step 18. Load new bitfile

• Download the fresh bitfile to the NetFPGA board:
  

nf2_download nf2_top_par.bit

Step 19. Run regression-test suite on new bitfile

• Re-run the regression test suite.
  

~/NF2/bin/nf21_regress_test.pl

• Sample correct output:
  

Running tests on project 'driver'...
: Running test 'driver_compile'... PASS
: Running test 'driver_install'... PASS
: Running test 'verify_mtu'... PASS

Running tests on project 'reference_router'...
:Running global setup... PASS
:Running test 'test_router_cpusend/run.pl'... PASS
:Running test 'test_wrong_dest_mac'... PASS
:Running test 'test_nonip_packet'... PASS
:Running test 'test_nonipv4_packet'... PASS
:Running test 'test_invalidttl_packet'... PASS
:Running test 'test_lpm_misses'... PASS
:Running test 'test_arp_misses'... PASS
:Running test 'test_badipchecksum_packet'... PASS
:Running test 'test_ipdest_filter_hit'... PASS
:Running test 'test_packet_fowarding'... PASS
:Running test 'test_lpm'... PASS
:Running test 'test_lpm_next_hop'... PASS
:Running test 'test_queue_overflow'... PASS
:Running test 'test_oq_limit'... PASS
:Running test 'test_ipdest_filter'... PASS

Running tests on project 'scone'...
:Running global setup... PASS
:Running test 'test_build'... PASS
:Running test 'test_mac_set'... PASS
:Running test 'test_ip_set'... PASS
:Running test 'test_rtable_set'... PASS
:Running test 'test_disabled_interfaces/run.pl'... PASS
:Running test 'test_noniparp_ethtype'... PASS
:Running test 'test_arp_rpl'... PASS
:Running test 'test_arp_norpl'... PASS
:Running test 'test_arp_quepkt'... PASS
:Running test 'test_ip_error'... PASS
:Running test 'test_ip_checksum/run.pl'... PASS
:Running test 'test_ttl_expired/run.pl'... PASS
:Running test 'test_send_receive/run.pl'... PASS
:Running test 'test_arp_req/run.pl'... PASS
:Running test 'test_tcp_port/run.pl'... PASS
:Running test 'test_udp_packet/run.pl'... PASS
:Running test 'test_unknown_ip/run.pl'... PASS
:Running test 'test_icmp_echo/run.pl'... PASS
:Running test 'test_icmp_notecho/run.pl'... PASS
:Running global teardown... PASS

Running tests on project 'gui_scone'...
:Running global setup... PASS
:Running test 'test_main_frame'... PASS
:Running test 'test_routing_table'... PASS
:Running test 'test_arp_table'... PASS
:Running test 'test_port_config_table'... PASS
:Running global teardown... PASS