Decache Forwarder

Show Contents...Hide Contents...

Contents

• Port Numbering
• Masks
• Perl Code
• LiveCD
• Files

Port Numbering

These are notes for using the Decache forwarder:

• The NetFPGA has 8 input/output Queues.
• i/o queues 0,2,4,6 are connected to the Ethernet ports 0,1,2,3
• i/o queues 1,3,5,7 are connected to the CPU interfaces nf2c0/1/2/3
• Packets that arrive to Ethernet queue 2i and need to go to software go to queue 2i+1 and are received on nf2ci
• Packets sent from nf2ci go out on queue 2i to the Ethernet port i.

This affects the way entries are written. Input ports are binary-encoded while outputs are specified as a bitmask .

For example, if an entry requires input queue 2 (i.e. Ethernet port 1), then the input port written should be 2. If the output is queue 4 (i.e. Ethernet port 2) then the output written should be 1<<4. The output mask thus allows multiple bits to be set to 1 to do multicast.

Masks

• For rule masks, a 1 in a bit position specifies a don't care meaning that bit will match anything.
• For replacement masks, a 1 bit specifies that a replacement should be done on that bit using the supplied replacement data. If a bit in the replacement mask is 0, the corresponding bit in a packet is not replaced.

Perl Code

Here is the API used in Perl:

 # bytes consists of:
 #  72 bytes rule
 #  72 bytes rule_mask
 #  72 bytes replacement data
 #  72 bytes replacement mask
 # The first 8 bytes corresponds to the IOQ module hdr
 # The rest correspond to the pkt bytes.
 sub insertEntry {
   my $idx  = shift;
   die "bytes input should be at least 72*4=288 bytes. Found ".scalar @_ if(scalar @_ != 72*4);
   die "Index should be between 0 and 31. Found $idx" unless ($idx >=0 && $idx < 32);
   print "insertEntry: to index $idx\n";
   my $cnt=0;
   foreach (@_) {
     printf("$cnt-%02x ", $_);
     $cnt++;
   }
   # write rule
   for(my $i=0; $i<9; $i++) {
     my $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();

my $addr = $i*512 + $idx*16 + main
WILDCARD_LOOKUP_BASE_REG(); NF2
PacketGen
nf_PCI_write32($addr, $word);

     printf "  Writing %08x into %08x\n", $word, $addr;
     $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();
     $addr += 4;

NF2
PacketGen
nf_PCI_write32($addr, $word);

     printf "  Writing %08x into %08x\n", $word, $addr;
   }
 
   # write rule_mask
   for(my $i=0; $i<9; $i++) {
     my $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();

my $addr = $i*512 + 8 + $idx*16 + main
WILDCARD_LOOKUP_BASE_REG(); NF2
PacketGen
nf_PCI_write32($addr, $word);

     printf "  Writing %08x into %08x\n", $word, $addr;
     $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();
     $addr += 4;

NF2
PacketGen
nf_PCI_write32($addr, $word);

     printf "  Writing %08x into %08x\n", $word, $addr;
   }
 
   # write data and data_mask
   for(my $i=0; $i<18*2; $i++) {
     my $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();

my $addr = $idx*512 + $i*4 + main
WILDCARD_ACTION_BASE_REG(); NF2
PacketGen
nf_PCI_write32($addr, $word);

     printf "  Writing %08x into %08x\n", $word, $addr;
   }
 
   # reset counters and timestamp

my $addr = $idx*512 + main
WILDCARD_ACTION_BASE_REG() + main
WILDCARD_LOOKUP_PKTS_HIT_SKIP(); NF2
PacketGen
nf_PCI_write32($addr, 0); NF2
PacketGen
nf_PCI_write32($addr+4, 0); NF2
PacketGen
nf_PCI_write32($addr+8, 0);

   printf "  Writing 0 into %08x\n", $addr;
   printf "  Writing 0 into %08x\n", $addr+4;
   printf "  Writing 0 into %08x\n", $addr+8;
 
   return 6*9+3;
 }
 
 sub readEntry {
   my $idx  = shift;
   die "bytes input should be at least 72*4=288 bytes. Found ".scalar @_ if(scalar @_ != 72*4);
   die "Index should be between 0 and 31. Found $idx" unless ($idx >=0 && $idx < 32);
 
   # read rule
   for(my $i=0; $i<9; $i++) {
     my $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();

my $addr = $i*512 + $idx*16 + main
WILDCARD_LOOKUP_BASE_REG(); NF2
PacketGen
nf_PCI_read32($addr, $word);

     $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();
     $addr += 4;

NF2
PacketGen
nf_PCI_read32($addr, $word);

   }
 
   # read rule_mask
   for(my $i=0; $i<9; $i++) {
     my $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();

my $addr = $i*512 + 8 + $idx*16 + main
WILDCARD_LOOKUP_BASE_REG(); NF2
PacketGen
nf_PCI_read32($addr, $word);

     $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();
     $addr += 4;

NF2
PacketGen
nf_PCI_read32($addr, $word);

   }
 
   # read data and data_mask
   for(my $i=0; $i<18*2; $i++) {
     my $word = (shift() << 24) + (shift() << 16) + (shift() << 8) + shift();

my $addr = $idx*512 + $i*4 + main
WILDCARD_ACTION_BASE_REG(); NF2
PacketGen
nf_PCI_read32($addr, $word);

   }
 
   return 6*9+3;
 }
 
 sub readCounters {
   my $idx = shift;
   my $pkt_count = shift;
   my $byte_count = shift;
 

my $addr = $idx*512 + main
WILDCARD_ACTION_BASE_REG() + main
WILDCARD_LOOKUP_PKTS_HIT_SKIP(); NF2
PacketGen
nf_PCI_read32($addr, $pkt_count); NF2
PacketGen
nf_PCI_read32($addr+4, $byte_count);

 }

Here is an example of how it is used:

my $pkt = NFUtils
SimplePacket->new(NFUtils
SimplePacket
PKT_TYPE() => NFUtils
SimplePacket
PKT_TYPE_UDP());

 my @pkt_bytes = $pkt->bytes();
 
 my $input_queue = 2;
 my $output_queue = 4;
 
 my @rule = @pkt_bytes[0..63];
 my @rule_mask = (0xff) x 64;
 my @pkt_replacement = (1..64);
 my @replacement_mask = ((0xff) x 32, (0x00) x 16, (0xff) x 16);
 my $idx = 2;
 
 my $entry = [$idx,
 	     # rule
 	     0, 0, 0, 0, 0, $input_queue, 0, 0, # IOQ hdr
 	     @rule,
 	     # rule mask
 	     0xff, 0xff, 0xff, 0xff, 0, 0, 0xff, 0xff, # IOQ hdr
 	     @rule_mask,
 	     # replacement
 	     0, (1 << $output_queue), 0, 0, 0, 0, 0, 0, # IOQ hdr
 	     @pkt_replacement,
 	     # replacement mask
 	     0xff, 0xff, 0, 0, 0, 0, 0, 0, # IOQ hdr
 	     @replacement_mask,
 	    ];
 

DecacheLib
insertEntry(@{$entry});

 my $input_port = ($input_queue/2) + 1;
 my $output_port = ($output_queue/2) + 1;
 
 # transform pkt
 my @exp_bytes = ();
 my $i;
 for($i=0; $isize()-1];

LiveCD

If you want to get on with stuff quickly, got to LiveCD, burn a copy and boot from it. It should have all the utilities you need.

Files

You can download a tarball from here This tarball has:

• bitfile
• DecacheLib.pm with functions used for simulation
• make_pkts.pl that uses DecacheLib.pm
• .h files for C development
• .ph files for Perl development