lxMicWorkload

Copyright (c) 2012-2013 Intel Corporation. All rights reserved.

version_number = 2.08.1.0001
Released:	 09/10/2014

Purpose
The purpose of this test module is to stress the CPU cores of a Many Integrated Cores (MIC) card installed in a 
system.

Runtime Environment
The Intel(R) Modular Test Architecture Test Executive for Linux.

License Agreement
All code is under the Intel(R) software License which can be found at http://legal.intel.com. A copy of this agreement 
must be included if code is released outside Intel.

Hardware Description
This module connects to the . 




Overview

Version Information
Version Information: V 2.08.1.0001
Supported Platform: Server
 
Purpose and Scope
Stress CPU cores on MIC card.

This test connects to the MIC Workloads software stack and runs tests in it, while also connecting to the CTC test 
executive for error handling.
 
Test Environment Requirements and Limitations
*	Requires Many Integrated Cores (MIC) card installed on system.
*	MPSS software stack must be loaded and installed as noted below.
*	Performance Workloads files must be downloaded and installed as 	noted in "Installation Instructions".
*	MPSS service must be started.

See "Installation Instructions" for detailed setup information.



Change History

Release Date	Version #		SW Developer	Change Description
7/20/2012		2.01.1.0001		SLK		First release
7/30/2012		2.02.1.0001		SLK		Added stdout and
									stderr redirection to
									script so user  
									can retrieve data if
									failure occurs. 
									Additional
									configuration
									information for the 
									user.
12/18/2012		2.03.1.0001		SLK		Linked to updated
									version of Workload 
									Report stdout &
									stderr msg in error
									data if available. 
									
05/23/2013		2.05.1.0001		SLK		Updated micprun in
									micperf tar file
10/03/2013		2.06.1.0001		DKJ		Update micprun in
									micperf tar file for
									MPSS 3.1 release.
12/05/2013		2.07.1.0001		DKJ		Added support for:
									*	Running MicPerf 										PCIe SHOC tests
									*	Testing more
										than one MIC
										card
									*	Updated to use
										MicPerf versions
										shipped with
										MPSS 3.1 release

10/10/2014		2.08.1.0001		DKJ		Updated to work with
									MPSS 3.3 release



Error Codes


0x523XX001	Error retrieving parameter 'PARAMETER_NAME'
0x523XX002	Invalid test name (TEST_NAME). Acceptable values: sgemm, dgemm, linpack
0x523XX003	Invalid margin value (VALUE): Must be a floating point value less than 1.0
0x523XX004	Invalid value (X) for parameter 'PARAMETER_NAME': Must be between Y and Z.
0x523XX005	Could not find 'DIRECTORY_NAME' directory
0x523XX006	Could not find 'FILE_NAME' file
0x523XX007	Internal error: ERROR_MSG
0x523XX008	Error running:
		Script: SCRIPT_NAME
		Flag file: FLAG_FILE_NAME
		Debug Data (Return Code and Msgs): RETURN_CODE
		MESSAGE
0x523XX009	Unable to read data from file FILE_NAME
0x523XX00A	Unable to create directory DIRECTORY_NAME
0x523XX00B	Unable to allocate memory
0x523XX00C	Unable to open file FILE_NAME
0x523XX00D	First matrix size (X) cannot be greater than last matrix size (Y)
0x523XX00E	Unable to remove file FILE_NAME. Return code X (MSG)
0x523XX00F	Script 'SGEMM' timed out after X seconds (MSG IF AVAILABLE)
0x523XX010	Please reset MIC card - unable to kill process X
0x523XX011	Killed process X. Please reset MIC card."
0x523XX012	uOS Boot failed on mic_
0x523XX013	MIC device mic_ not available <MPSS API Function>: <MPSS API Error Message>
0x523XX014	Error Setting <Parameter Name> to <Parameter Value> for <MIC Device Name>
0x523XX015	MicPerf Environment Setup Error. Check lxMicWorkload setup documentation.

Installation Instructions



Install MPSS Software Stack:
 Uninstall the old MPSS RPMs: See MPSS Software documentation for information on how to uninstall MPSS RPMs.

- Note: Version 2.08.1.0001 of lxMicWorkload requires MPSS version 3.3.
 
 Install new MPSS RPMs: See MPSS Software documentation for information on how to install the new MPSS RPMs.  
Include the MicPerf RPMs in addition to the normal MPSS rpms.  The MicPerf RPMs are: 
micperf-<version>.build0.glib2.12.2.x86_64.rpm and micperf-data-<vesion>.build0.glicb2.12.2.x86_64.rpm.  
These rpms can normally be found in the perf subdirectory for a complete MPSS release.

-  After the MicPef RPMs are installed look in: /usr/share/doc/micperf-<mpss_version>/INSTALL.TXT for detailed 
MicPerf Install instructions. For running with CTC and lxMicWorkloads, the following MicPerf isntall/setup process can 
be used:
*	Download Intel C++ Composer XE 2013 for LInux package from:
 
http://software.intel.com/en-us/articles/redistributable-libraries-for-the-i
ntel-c-and-fortran-composer-xe-2013-for-linux
*	The Composer XE redistributable package must match the version used to build the MicPerf release. In the case of  
MPSS 3.3, this is Update 2 (i.e. version 2.146).
*	Install Composer XE redistributable package on test host, by untaring the package (e.g. 
l_ccompxe_2013.2.146.redist.tgz) and runnig the included install.sh script.

Test Host Root User Environment Setup:
*	MicPerf execution depends on including MicPerf directory paths, in some BASH environment variables, and 
initialization of the Composer XE environment.
*	This setup must be done for every BASH shell instance which will execute the MicPerf tests, including the BASH 
shells created by lxMicWorkload.
*	The easiest way to do this is to add the following lines at the end of /root/.bashrc  (assuming the root user will be 
running these tests): Note: With MPSS 3.3 soft links to the micperf executables are made in /usr/bin, so PATH 
does not need to be modified to include a micperf directory.

# Setup micperf PYTHONPATH requirements:
export PYTHONPATH=/usr/src/micperf/micp:${PYTHONPATH}

# Setup Composer_XE redist, which is required to run MicPerf benchmarks
source /root/bin/compilervars.sh ia32

Setup Passwordless SSH access from Test Host to uOS's running on MIC devices:
 
*	Make sure MPSS service is stopped: 	service mpss stop
*	Create ssh key with no parameters:	ssh-keygen
*	Reset ssh configuration:		micctrl --resetconfig
*	Reset ssh defaults:			micctrl --resetdefaults
*	Reboot host: 			shutdown -r now
*	Start MPSS service: 			service mpss start
*	Remove old known_hosts file:		rm /root/.ssh/known_hosts
		(This assumes you are root - if another username, replace "root" with your user name.)
		(If you have other ssh setup you need to preserve, just remove the mic0 line from known_hosts). 	
*	Connect to each MIC card (i.e. mic0, mic1, etc) to add it to known_hosts. For example, if this is mic device 0, run:
	ssh mic0
It will say that the authenticity of the host can't be established, and ask if you  want to continue connecting. 
Respond with "yes".

If issues occur later (in particular, a printf message in the terminal where CTC was invoked will say "The authenticity 
of host 172.31.X.X (172.31.X.X) cant be established. RSA key fingerprint is xx:xx:Are you sure you wish to 
continue connecting (yes/no)?:, check a few things:

1. Make sure <home directory>/.ssh/id_rsa file has only "user" read and write privileges set. Example: ls -al 
/root/.ssh/id_rsa

2. If the connection to the mic card becomes corrupted, you may need to remove the line referencing that card from 
the known_hosts file. If you are running as root, this can be found at /root/.ssh/known_hosts. If you are running as an 
individual user, check for the .ssh directory inside your /home/<username> directory. As with all directories starting 
with .ssh, it is invisible by default - you have to change directories into it.

Subtests


Test 0	Auto-Configuration

Test 1	SGEMM

Test 2	SHOC Download

Test 3	SHOC Readback



Test0 - Auto-Configuration

Overview

Auto configure will walk through all groups under the lxMicWorkload device (for example, SGEMM), checking to see 
if the necessary MIC OEM Workloads files exist, if the MIC device is plugged into the host and the MPSS software 
stack is booted. It will then run the designated test (for example, SGEMM) using user-defined parameters on OEM 
Workloads and store the results. Later tests will be compared against these results.

IMPORTANT: Do not manually cancel this test by pressing STOP button unless absolutely necessary. If required, 
restart system before restarting tests. Otherwise, tests will not work as expected.

Parameter(s)

The following parameters are used or written by this test:

micperf_device

sgemm_first_matrix_size

sgemm_last_matrix_size

micperf_margin

sgemm_num_threads

micperf_output_dir

micperf_output_tag

sgemm_rep

micperf_script_timeout

micperf_stderr_redirect_file 
 
micperf_stdout_redirect_file 
 
sgemm_step

shoc_nopinned

shoc_passes


Test Execution Time

Depends on test configuration user has chosen - it will vary between ~30 seconds to several minutes.

Dependencies/Limitations

	See "Installation Instructions"



Test1 - SGEMM

Overview

This test will check to see if the necessary MIC OEM Workloads files exist, if the MIC device is plugged into the host 
and the MPSS software stack is booted. It will then run the MicPerf SGEMM test, on the target MIC device, with 
user-specified parameters. The results of this test run are compared to the results stored during Auto-Configuration. 
If the results differ by more than the value specified in the margin parameter, the test will fail. Otherwise, the test will 
pass.

The SGEMM test will create an array of size micperf_first_matrix_size and run a SGEMM algorithm on it. If the value 
stored in micperf_last_matrix_size is greater than that stored in micperf_first_matrix size, the test will run multiple 
times, each time creating an array of size micperf_first_matrix_size + (micperf_step * i), until reaching the size 
specified in micperf_last_matrix_size.

IMPORTANT: Do not manually cancel this test by pressing STOP button unless absolutely necessary. If required, 
restart system before restarting tests. Otherwise, tests will not work as expected.

Parameter(s)

The following parameters are used by this test:

micperf_device

sgemm_first_matrix_size

sgemm_last_matrix_size

micperf_margin

sgemm_num_threads

micperf_output_dir

micperf_output_tag

sgemm_rep

micperf_script_timeout

micperf_stderr_redirect_file 
 
micperf_stdout_redirect_file 
 
sgemm_step


Test Execution Time

Depends on test configuration user has chosen - it will vary between ~30 seconds to several minutes.

Dependencies/Limitations

See "Installation Instructions"

Test2 - SHOC Download

Overview

This test will check to see if the necessary MIC OEM Workloads files exist, if the MIC device is plugged into the host 
and the MPSS software stack is booted. It will then run the MicPerf SHOC Download test, on the target MIC device, 
with user-specified parameters. The results of this test run are compared to the results stored during 
Auto-Configuration. If the results differ by more than the value specified in the margin parameter, the test will fail. 
Otherwise, the test will pass.

The SHOC Download test will run on the MIC device, the number of time specified by shoc_passes. The parameter 
shoc_nopinned can be used to control whether the SHOC Download test, running on the MIC device will pin 
memory pages in the MIC uOS memory.



Parameter(s)

The following parameters are used by this test:
	
	micperf_device
	micperf_margin
	micperf_output_dir
	micperf_script_timeout
	micperf_stderr_redirect_file
	micperf_stdout_redirect_file
	micperf_tag_current
	micperf_tag_master
	shoc_passes
	shoc_nopinned

Test Execution Time

Depends on test configuration user has chosen - it will vary between ~30 seconds to several minutes.

Dependencies/Limitations

See "Installation Instructions"

Test3 - SHOC Readback

Overview

This test will check to see if the necessary MIC OEM Workloads files exist, if the MIC device is plugged into the host 
and the MPSS software stack is booted. It will then run the MicPerf SHOC Readback test, on the target MIC device, 
with user-specified parameters. The results of this test run are compared to the results stored during 
Auto-Configuration. If the results differ by more than the value specified in the margin parameter, the test will fail. 
Otherwise, the test will pass.

The SHOC Readback test will run on the MIC device, the number of time specified by shoc_passes. The parameter 
shoc_nopinned can be used to control whether the SHOC Download test, running on the MIC device will pin 
memory pages in the MIC uOS memory.



Parameter(s)

The following parameters are used by this test:
	
	micperf_device
	micperf_margin
	micperf_output_dir
	micperf_script_timeout
	micperf_stderr_redirect_file
	micperf_stdout_redirect_file
	micperf_tag_current
	micperf_tag_master
	shoc_passes
	shoc_nopinned


Test Execution Time

Depends on test configuration user has chosen - it will vary between ~30 seconds to several minutes.

Dependencies/Limitations

See "Installation Instructions"

Parameters


micperf_device

micperf_margin

micperf_output_dir

micperf_output_tag

micperf_script_timeout
topic_sgemm_first_matrix_size
sgemm_first_matrix_size

sgemm_last_matrix_size
topic_sgemm_step
sgemm_step

sgemm_rep

sgemm_num_threads

shoc_passes

shoc_nopinned

micperf_device

Description

This parameter indicates which device number the tests should be run on. The device number is assigned by the 
MPSS software stack as it boots, and is zero-based (the first device found will be device 0, the second will be device 
1, etc.).

Default

	micperf_device = 0

Dependencies

MIC card must be installed on host.
MPSS Software stack must be booted.

Used by

Auto-Configure
SGEMM

See Also

Parameters

sgemm_first_matrix_size

Description

This parameter is used only for the SGEMM test. It indicates the starting point of the first matrix used in the SGEMM 
algorithm.

This parameter is optional, and if not specified, the default value will be used.

These three parameters are closely tied: sgemm_first_matrix_size, sgemm_last_matrix_size, and sgemm_step. One 
common default scenario would be:
sgemm_first_matrix_size = 1024
sgemm_last_matrix_size = 4096
sgemm_step = 256

In this case, multiple test matrixes from 1024 to 4096 will be created, separated by jumps of 256: 1024, 1280, 1536, 
... 3584, 3840, 4096.

Default

	sgemm_first_matrix_size = 1024

Dependencies

*	sgemm_first_matrix_size must be less than or equal to sgemm_last_matrix_size.
*	If you change the values of sgemm_first_matrix_size, sgemm_last_matrix_size, sgemm_step or sgemm_rep, you 
should run Auto-Configuration again. Otherwise, you will be comparing different test runs, and the test results will 
differ accordingly.

Used by

Auto-Configure
SGEMM

See Also

Parameters

sgemm_last_matrix_size

Description

This parameter is used only for the SGEMM test. It indicates the ending point of the matrixes used in the SGEMM 
algorithm.

This parameter is optional, and if not specified, the default value will be used.

These three parameters are closely tied: sgemm_first_matrix_size, sgemm_last_matrix_size, and sgemm_step. One 
common default scenario would be:
sgemm_first_matrix_size = 1024
sgemm_last_matrix_size = 4096
sgemm_step = 256

In this case, multiple test matrixes from 1024 to 4096 will be created, separated by jumps of 256: 1024, 1280, 1536, 
... 3584, 3840, 4096.

Default

	sgemm_last_matrix_size = 4096

Dependencies

*	sgemm_last_matrix_size must be greater than or equal to sgemm_first_matrix_size.
*	If you change the values of sgemm_first_matrix_size, sgemm_last_matrix_size, micperf_step or sgemm_rep, you 
should run Auto-Configuration again. Otherwise, you will be comparing different test runs, and the test results will 
differ accordingly.

Used by

Auto-Configure
SGEMM

See Also

Parameters

micperf_margin

Description

This value indicates the margin within which the results of the auto-configured test and the test currently running 
must fall in order to pass.  Value must be a numeric value greater than 0.0 and less than 1.0. 

0.20 indicates 20%, 0.04 indicates 4%, etc.

Default

	micperf_margin = 0.20

Dependencies

	None

Used by

Auto-Configure
SGEMM


See Also

Parameters

sgemm_num_threads

Description

Number of threads which the SGEMM test should execute on. For example, a 52 core MIC card with four threads per 
core would have 208 threads.

Default

	sgemm_num_threads = 52

Dependencies

Do not set threads to greater than the available threads on the MIC card.

Used by

Auto-Configure
SGEMM

See Also

Parameters

micperf_output_dir

Description

This is the base name for the subdirectories, in the CTC directory, where MicPerf test results will be stored, including 
the master results files created during autoconfig. Using  this base name, one directory will be created for each MIC 
device tested.  For example, in a system with two MIC devices (i.e. mic0 and mic1), when this parameter is set to 
the default value of of ./micperf two output directories will be created, named: ./micperf_mic0 and ./micperf_mic1 
All output files should be written to these directories. This includes the comparison file written by Auto-Configuration, 
and the file for the current test.



Default

	micperf_output_dir = ./micperf

Dependencies

Comparison file saved by auto-configuration must be saved to this directory.

Used by

Auto-Configure
SGEMM


See Also

Parameters

micperf_tag_current

Description

This value is used, along with the test name (e.g. sgemm, shoc_Download, etc)  in the creation of the file name for 
saving results of this current run. For example, if the current tag is called current the SGEMM test output file will be 
titled micp_run_stats_sgemm_current.pkl.

Default

	micperf_tag_current = current

Dependencies

micperf_tag_current and micperf_tag_master must be different values.

Used by

Auto-Configure
SGEMM


See Also

Parameters

micperf_tag_master

Description

This value is used in the creation of the file name for saving results of the master run. For example, if the master tag 
is called "sgemm_master", the master output file will be titled micp_run_stats_sgemm_master.pkl.
This value is used, along with the test name (e.g. sgemm, shoc_Download, etc)  in the creation of the file name for 
saving results of the master (i.e. AutoConfig) run. For example, if the master tag is called master the SGEMM 
master test output file will be titled micp_run_stats_sgemm_master.pkl.

Default

	micperf_tag_master = master

Dependencies

micperf_tag_current and micperf_tag_master must be different values.

Used by

Auto-Configure
SGEMM


See Also

Parameters

sgemm_rep

Description

This value indicates the number of times the MicPerf SGEMM tests should loop internally. The higher this value, the 
more times the test will run, which means the test time will be longer. With higher repetitions the performance 
statistics are also more more accurate, as they're averaged across test runs.

Default

	sgemm_rep = 32

Dependencies

This must be a value between 1 and 32.

Used by

Auto-Configure
SGEMM


See Also

Parameters

micperf_script_timeout

Description

This value indicates the number of seconds to let the micprun script run before assuming that the test has stopped 
responding. When this occurs, the test will fail. The test waits for the creation of the flag file as set in 
micperf_flag_file. Once the file exists, micprun is assumed to have completed.

If running an extremely lengthy test, both the "timeout" value in the CTC device (right click on the device and 
choose "Properties") and the micperf_script_timeout in this test must be increased to larger than the default test 
time. Otherwise the script will time out. The default timeout values are fine for the sample tests, however.

Default

micperf_script_timeout = 180

Dependencies

None

Used by

Auto-Configure
SGEMM


See Also

Parameters

micperf_stderr_redirect_file

Description

This value indicates the base file name used to create MIC device specific output files, to which all standard error 
(stderr) output is stored. This base file name is customized for each MIC device by adding the MIC device name to 
the beginning of the base name.
For example if micperf_stderr_redirect_file is stderr_redirect.txt  standard error output from tests run on MIC0 
will be sent to mic0_stderr_redirect.txt. In the case of a OEM Workload Failure, some data may be stored to this 
file. Other data will be stored at the location specified by the micperf_stdout_redirect_file parameter.

File will be output in current directory.

Default

micperf_stderr_redirect_file = stderr_redirect.txt

Dependencies

None

Used by

Auto-Configure
SGEMM


See Also

Parameters

micperf_stdout_redirect_file

Description

This value indicates the base file name used to create MIC device specific output files, to which all standard 
(stdout) output is stored. This base file name is customized for each MIC device by adding the MIC device name 
to the beginning of the base name.
For example if micperf_stdout_redirect_file is stdout_redirect.txt  standard output from tests run on MIC0 will 
be sent to mic0_stdout_redirect.txt. In the case of a OEM Workload Failure, some data may be stored to this 
file. Other data will be stored at the location specified by the micperf_stderr_redirect_file parameter.


File will be output in current directory.

Default

micperf_stdout_redirect_file = stdout_redirect.txt

Dependencies

None

Used by

Auto-Configure
SGEMM


See Also

Parameters

sgemm_step

Description

This parameter is used only for the SGEMM test. It indicates the steppings between multiple iterations of the 
matrixes used in the SGEMM algorithm. 

This parameter is optional, and if not specified, the default value will be used.

These three parameters are closely tied: sgemm_first_matrix_size, sgemm_last_matrix_size, and sgemm_step. One 
common default scenario would be:
sgemm_first_matrix_size = 1024
sgemm_last_matrix_size = 4096
sgemm_step = 256

In this case, multiple test matrixes from 1024 to 4096 will be created, separated by jumps of 256: 1024, 1280, 1536, 
... 3584, 3840, 4096.

Default

	sgemm_step = 256

Dependencies

*	If you change the values of sgemm_first_matrix_size, sgemm_last_matrix_size, sgemm_step or sgemm_rep, you 
should run Auto-Configuration again. Otherwise, you will be comparing different test runs, and the test results will 
differ accordingly.

Used by

Auto-Configure
SGEMM


See Also

Parameters

shoc_passes

Description

This parameter is used by the SHOC tests (SHOC Download and SHOC Readback). It indicates the times the SHOC 
test should be executed on the target MIC uOS. 

This parameter is optional, and if not specified, the default value will be used.


Default

	shoc_passes = 10

Dependencies

*	If you change the value of shoc_passes, you should run Auto-Configuration again. Otherwise, you will be 
comparing different test runs, and the test results will differ accordingly.

Used by

Auto-Configure
SGEMM


See Also

Parameters

shoc_nopinned

Description

This parameter is used by the SHOC tests (SHOC Download and SHOC Readback). It is used to control whether the 
SHOC test will pin pages in memory when it is executing.  
This parameter is optional, and if not specified, the default value will be used.


Default

	shoc_nopinned = true

Dependencies

*	If you change the value of shoc_pinned, you should run Auto-Configuration again. Otherwise, you will be 
comparing different test runs, and the test results will differ accordingly.

Used by

Auto-Configure
SGEMM


See Also

Parameters




	 
