Asymmetric Multi Processing (AMP) enables a multiprocessor system to run multiple operating systems (OS) that are separate from one another. In other words, each CPU has its own private memory space that contains both the operating system and the applications that will run on it. In addition, there may be some shared memory space utilized for multiprocessor communications. In contrast, Symmetric Multiprocessing (SMP) allows a single operating system to run on many CPUs while sharing a public memory space.
This tutorial is the result of recent work with the Freescale P4080 Processor. If you’re not familiar with this chip, it’s an 8-core beast from the PowerPC family. (The manual has almost 3500 pages!)
The figure below shows a typical AMP system. Once CPU0 boots up, the wrload utility is used to load a specific vxWorks image from the file system into the target CPU’s memory area. CPU 0 has a larger percentage of the available memory space, while the remaining CPUs have a smaller area to work with.
There are also two types of AMP: supervised (hypervisor) and unsupervised. This post will solely cover unsupervised AMP. Hypervisor is a whole different topic.
Why Use AMP? #
AMP is well-suited to applications that can be divided into logical modules (such as control plane and data plane processing in a router), or where an increased level of security is required on some processors (devices are assigned to processors, allowing the MMU to separate IO connections and secure applications). Furthermore, completely different operating systems can be installed into the CPUs, thus you may want to use a combination of Linux and other versions of vxWorks.
AMP is also useful for modernizing older systems by merging many boards onto a single board. The system design can remain same, and porting is made much easier by taking an old application that operated on an old board and moving it to one of the CPUs in a multiprocessor system. Using AMP eliminates the requirement to integrate all application code into a single code space. MIPC Network Devices (MND) can even preserve existing Ethernet connections without requiring any adjustments.
Building AMP in VxWorks #
You have two methods for creating VxWorks images for AMP: utilizing WindRiver Workbench (the GUI) or the command line. The disadvantage of utilizing Workbench is that for eight CPUs, you must configure eight VxWorks images individually, which can be time-consuming. Instead, write a makefile similar to this one under /workspace:
###############################################################################
#
# makefile for amc718 AMP builds
#
# This makefile demonstrates how to build an AMP system for the
# Vadatech AMC718 board. This board uses an 8 core Freescale P4080 Chip
#
# From a vxWorks development shell, type make amp. VxWorks images will be copied
# into FTP_DIR
#
# for Windows hosts, fix slashes
WIND_HOME := $(subst \,/,$(WIND_HOME))
WIND_BASE := $(subst \,/,$(WIND_BASE))
WIND_USR := $(subst \,/,$(WIND_USR))
TOOL = gnu
# all vxWorks images will get copied here for pickup
FTP_DIR = C:/temp/incoming
AMP0_DIR = amc718_gnu_amp0
AMP0_PROJ = $(AMP0_DIR)/$(AMP0_DIR).wpj
AMP1_DIR = amc718_gnu_amp1
AMP1_PROJ = $(AMP1_DIR)/$(AMP1_DIR).wpj
AMP2_DIR = amc718_gnu_amp2
AMP2_PROJ = $(AMP2_DIR)/$(AMP2_DIR).wpj
AMP3_DIR = amc718_gnu_amp3
AMP3_PROJ = $(AMP3_DIR)/$(AMP3_DIR).wpj
AMP4_DIR = amc718_gnu_amp4
AMP4_PROJ = $(AMP4_DIR)/$(AMP4_DIR).wpj
AMP5_DIR = amc718_gnu_amp5
AMP5_PROJ = $(AMP5_DIR)/$(AMP5_DIR).wpj
AMP6_DIR = amc718_gnu_amp6
AMP6_PROJ = $(AMP6_DIR)/$(AMP6_DIR).wpj
AMP7_DIR = amc718_gnu_amp7
AMP7_PROJ = $(AMP7_DIR)/$(AMP7_DIR).wpj
amp: amp0 amp1 amp2 amp3 amp4 amp5 amp6 amp7
amp0:
vxprj create -force $(BSP) $(TOOL) $(AMP0_PROJ)
vxprj bundle add $(AMP0_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP0_PROJ) BUNDLE_AMP_PRI
vxprj component add $(BSP_PROJ) INCLUDE_PCI_BUS
vxprj build $(AMP0_PROJ)
cp $(AMP0_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.0
amp1:
vxprj create -force $(BSP) $(TOOL) $(AMP1_PROJ)
vxprj bundle add $(AMP1_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP1_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP1_PROJ) INCLUDE_AMP_CPU_01
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP1_PROJ) INCLUDE_WDB_SYS
vxprj build $(AMP1_PROJ)
cp $(AMP1_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.1
amp2:
vxprj create -force $(BSP) $(TOOL) $(AMP2_PROJ)
vxprj bundle add $(AMP2_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP2_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP2_PROJ) INCLUDE_AMP_CPU_02
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP2_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices
vxprj component remove $(AMP2_PROJ) INCLUDE_NETWORK
vxprj build $(AMP2_PROJ)
cp $(AMP2_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.2
amp3:
vxprj create -force $(BSP) $(TOOL) $(AMP3_PROJ)
vxprj bundle add $(AMP3_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP3_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP3_PROJ) INCLUDE_AMP_CPU_03
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP3_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices
vxprj component remove $(AMP3_PROJ) INCLUDE_NETWORK
vxprj build $(AMP3_PROJ)
cp $(AMP3_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.3
amp4:
vxprj create -force $(BSP) $(TOOL) $(AMP4_PROJ)
vxprj bundle add $(AMP4_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP4_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP4_PROJ) INCLUDE_AMP_CPU_04
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP4_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP4_PROJ) INCLUDE_NETWORK
vxprj build $(AMP4_PROJ)
cp $(AMP4_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.4
amp5:
vxprj create -force $(BSP) $(TOOL) $(AMP5_PROJ)
vxprj bundle add $(AMP5_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP5_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP5_PROJ) INCLUDE_AMP_CPU_05
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP5_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP5_PROJ) INCLUDE_NETWORK
vxprj build $(AMP5_PROJ)
cp $(AMP5_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.5
amp6:
vxprj create -force $(BSP) $(TOOL) $(AMP6_PROJ)
vxprj bundle add $(AMP6_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP6_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP6_PROJ) INCLUDE_AMP_CPU_06
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP6_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP6_PROJ) INCLUDE_NETWORK
vxprj build $(AMP6_PROJ)
cp $(AMP6_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.6
amp7:
vxprj create -force $(BSP) $(TOOL) $(AMP7_PROJ)
vxprj bundle add $(AMP7_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP7_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP7_PROJ) INCLUDE_AMP_CPU_07
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP7_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP7_PROJ) INCLUDE_NETWORK
vxprj build $(AMP7_PROJ)
cp $(AMP7_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.7
Using wrload #
The wrload utility runs in the vxWorks shell on the first CPU (cpu0). It will load a vxWorks image from a mounted disk, transfer the code and data to the target CPU, and then zero the.bss segment. I discovered that it can be sensitive to successive loading, therefore I use taskDelays between calls. Also, vxWorks require a bootline and does not function well when one is missing. Consider the case below:
wrload ("-file host:/vxWorks.1 -cpu 1 -tsym \"*sysBootLine=dtsec(1,1) host:/vxWorks.1 h=192.168.1.100 e=192.168.1.51\" ");
taskDelay (60);
wrload ("-file host:/vxWorks.2 -cpu 2 -tsym \"*sysBootLine=dtsec(2,2) \" ");
taskDelay (60);
Using ‘tip’ to get a Shell on a CPU #
tip is a new function that connects to the other CPUs via an MSD (MIPC Serial Device), allowing a shell to be established even when there is no physical serial connection available. The BSP automatically creates MSD connections:
-> devs
drv name
0 /null
1 /tyCo/0
1 /tyCo/1
1 /ttyMsd0
1 /ttyMsd1
1 /ttyMsd2
1 /ttyMsd3
1 /ttyMsd4
1 /ttyMsd5
1 /ttyMsd6
To connect CPU1 and CPU2 (the first msd connection, msd0, will connect to cpu1):
tip ("dev=/ttyMsd0", "dev=/ttyMsd1")
~?
[Available commands]
~.: Drop connection and exit
~l: List available sessions
~s: Switch session
~?: Print command summary