NVMe-CLI
The nvme-cli has been modified to support SEF Units with the addition of the Kioxia plugin.
The nvme-cli allows the user to interact with a SEF Unit at a low level without using the library; however, it does not provide as much functionality or user friendliness as SEF-CLI.
After installing the SEF nvme-cli command, you will be able to execute the following commands.
usage: nvme kioxia <command> [<device>] [<args>]
The following are all implemented sub-commands:
sef-info SEF Unit Information
create-vd Create a Virtual Device
create-cap-cfg Create a Capacity Configuration
sel-cap-cfg Select a Capacity Configuration
delete-vd Delete a Virtual Device
create-qosd Create a QoS Domain
delete-qosd Delete a QoS Domain
setroot-qosd Set Rootpointer to QoS Domain
change-cap-qosd Change the capacity of a QoS Domain
change-weights-qosd Change the weights of a QoS Domain
read-deadline-qosd Change the read deadline for a QoS Domain
max-open-sb-qosd Change the max number of open Super Blocks on a QoS Domain
def-rq-qosd Change the read queue for a QoS Domain
vd-info Virtual Device Information
qd-info QosD Info
cap-cfg-list Capacity Config List
sb-info Super Block Information
sb-list Super Block List
ua-list User Address List
aco-list Address Change Order
rf-list-log Read Fifo List
rf-attach Attach Read Fifo to virtual domain
rf-detach Detach Read Fifo from virtual domain
vd-set-ndies Set the number of dies for a virtual domain
vd-set-pslc Set the pSLC blocks for a virtual domain
read Submit Read command, return results
write Submit FLA Request and NLW command, return results
copy Submit NLC command, return results
sb-manage Super Block Management
aen-rq Asynchronous event change request
SEF Info
Get the SEF Unit Information.
$ nvme kioxia sef-info /dev/nvme0
SEF Capability Supports:
Packed ADU Offset
List indicated Nameless Copy command
Super Block operated with Nameless Write and Nameless Copy
commands
pSLC Super Block
Number of Channels: 4
Number of Banks: 4
Number of Blocks per Die: 1958
Number of Pages per Block: 384
Number of Planes per Page: 6
Plane Data Size: 14
Expiration Open Period: 43200
Number of Placement IDs: 8
Max Number of Open Super Blocks: 2048
Number of Read FIFOs: 256
Number of Pages to Secure Integrity: 0
----DYNAMIC INFORMATION----
Number of Read FIFOs in Use: 0
Number of Virtual Devices: 0
Number of QoS Domains: 0
$
Virtual Device
Create
The create-vd command requires a Virtual Device descriptor file For example, this will create 1 Virtual Device spanning 4 dies:
cd <path to SEF_SDK>/lib
devtool/script_for_cli/gen_create_cap_cfg.sh -f VDFD.out -d "1 1 1 1"
nvme kioxia create-vd /dev/nvme0 -f
devtool/script_for_cli/data/VDFD.out
Info
Get the Virtual Device Information.
$ nvme kioxia vd-info /dev/nvme0 -v 1 -n 16
VDID 1
Critical Warning 0x0
CWC 7
CESN 0
MAX_PE 0
AVG_PE 0
NSBDIE 16
SBBW 23
AOBW 23
RPC 0
MXOSB 0
DRFID 1
NRF 1
NPBLK 0
VDCAP 72105984
VDGCAP 0
VDUCAP 0
ASS_QOSD 0
NDIE 15
DIES -
000000 000001 000002 000003 000004 000005 000006 000007
000008 000009 00000a 00000b 00000c 00000d 00000e
00000f
$
- VDID: Virtual Device ID
- CWC: Critical Warning Control
+ 7:3 - Reserved
+ 2 - Capacity Reduction Event
+ 1 - Out of pSLC Super Block Event
+ 0 - Out of general Super Block Event - CESN: Current Erase Serial Number
- MAX_PE: Maximum Program/Erase Count
- AVG_PE: Average Program/Erase Count
- NSBDIE: Number of Dies per Super Block
- SBBW: SuperBlock Bit Width
- AOBW: ADU Offset Bit Width
+ SBBW and AOBW determine the appearance of the SEFFlashAddress
+ Together they can add up to 64 or no less than 32-bits.
+ For example: SBBW = 16, AOBW = 32
+ Super Block bits: x, ADU Offset bits: y
+ SEFFlashAddress: 0000 xxxx yyyy yyyy - RPC: Recommended Patrol Cycle
- MAXOSB: Maximum Number of Open Super Blocks
- DRFID: Default Read FIFO ID
- NRF: Number of Read FIFOs
- NPBLK: Number of pSLC Blocks
- VDCAP: Virtual Device Total Capacity
- VDGCAP: Virtual Device Guaranteed Capacity
- VDUCAP: Virtual Device In-use Capacity
- ASS_QOSD: Number of QoS Domains assigned
- NDIE: Number of Dies
- DIES: List of Die IDs
Delete Virtual Device
Delete specific Virtual Device by ID or use 0xffff to delete all Virtual Devices.
nvme kioxia delete-vd /dev/nvme0 -v 0xffff
Create Capacity Configuration
Create a capacity configuration that assigns a Virtual Device ID to each die on the SEF Unit.
cd <path to SEF_SDK>/lib
devtool/script_for_cli/gen_create_cap_cfg.sh -f CAPCFG.ld2 -d "
1 1 2 2 2 2 0 0 \
1 1 2 2 2 2 0 0 \
1 1 3 3 3 3 0 0 \
1 1 3 3 3 3 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0 \
0 0 0 0 0 0 0 0";
nvme kioxia create-cap-cfg /dev/nvme0 -f
devtool/script_for_cli/data/CAPCFG.ld2
Cap Config List
$ nvme kioxia cap-cfg-list /dev/nvme0
CCI 1
DI 0
EGCN 16
ENDGID 1
CAF 0
EGSETS 0
EGCHANS 1
CI -1
CHMUS 1
MU 0, ML 0
...
$
- CCI: Capacity Configuration Index
- DI: Domain Identifier
- EGCN: Endurance Group Channel Number
- ENDGID: Endurance Group ID
- CAF: Capacity Adjustment Factor
- EGSETS: Number of NVM Sets
- EGCHANS: Number of Channels
- CI: Channel Identifier
- CHMUS: Number of Channel Media Units
- MU: Media Unit Identifier
- ML: Media Unit Descriptor Length
Select Cap Config
After creating a secondary capacity configuration, swap between them.
nvme kioxia sel-cap-cfg /dev/nvme0 -c 2
Set Number of Dies for Virtual Device
nvme kioxia vd-set-ndies /dev/nvme0 -v1 -n8
Set Number of pSLC blocks for a Virtual Device
nvme kioxia vd-set-pslc /dev/nvme0 -v1 -n10
QoS Domain
A Virtual Device should be created before the following actions can be performed.
Create QoS Domain
cd <path to SEF_SDK>/lib
devtool/script_for_cli/gen_create_qosd_bin.sh -v 1 -s 0 -p 8 -c 50000
-f QOSFD.out
nvme kioxia create-qosd /dev/nvme0 -q 1 -f
devtool/script_for_cli/data/QOSFD.out
Delete QoS Domain
nvme detach-ns /dev/nvme0 -n1 -c1
nvme kioxia delete-qosd /dev/nvme0 -q 1
QoS Domain Info
$ nvme kioxia qd-info /dev/nvme0 -q 1
QoS Domain Size: 288276504
QoS Domain Capacity: 288276504
QoS Domain Utilization: 0
Virtual Device ID: 1
Read Deadline: 0
Defect Management Type: 0
Number of PLIDs(0’s): 0
Max Open SuperBlocks: 0
Write Weight: 0
Erase Weight: 1
Default Read FIFO ID: 257
SuperBlock Statistics
Max Planes: 18449696256
Guaranteed Planes: 18449696256
Current Planes: 0
Active SuperBlocks: 0
Open SB for NLW: 0
Open SB for Erase: 0
pSLC SuperBlock Statistics
Max Planes: 0
Guaranteed Planes: 0
Current Planes: 0
Active SuperBlocks: 0
Open SB for NLW: 0
Open SB for Erase: 0
Root Pointer 0: 0x0
...
$
QoS Domain Capacity
nvme kioxia change-cap-qosd /dev/nvme0 -q1 -t0 -g4096
QoS Domain Weights (Read/Program)
nvme kioxia change-weights-qosd /dev/nvme0 -q1 -e100 -w100
Change QoS Domain Read Deadline
nvme kioxia read-deadline-qosd /dev/nvme0 -q1 -r2
Maximum Number of Open Super Blocks
nvme kioxia max-open-sb-qosd /dev/nvme0 -q1 -m256
QoS Domain Read Queue
nvme kioxia def-rq-qosd /dev/nvme0 -q1 -r8
Set Root QoS Pointer
nvme kioxia setroot-qosd /dev/nvme0 -q1 -f 0x0002000100000000 -i0
FIFO
A Virtual Device should be created before the following actions can be performed.
Attach a read FIFO
nvme kioxia rf-attach /dev/nvme0 -v 1 -f 2 -w 0x32
Detach a read FIFO
nvme kioxia rf-detach /dev/nvme0 -v 1 -f 2 -w 0x32
List available read FIFOs
$ nvme kioxia rf-list /dev/nvme0 -v1 -n128
Fifo Id = 0000000001, VD Id = 000001, Weight = 000032
Fifo Id = 0000000002, VD Id = 000002, Weight = 000064
...
$
List a specific read FIFO
$ nvme kioxia rf-list-log /dev/nvme0 -v 1 -f 1 -l 8
Fifo Id = 0000000001, VD Id = 000001, Weight = 000032
$
Super Block
An available Super Block is required to perform the following commands.
Super Block Info
$ nvme kioxia sb-info /dev/nvme0 -v 1 -s 10
VDID : 1
ST : <State>
SBS : NLW (04)
SBID : 0x00000001
PID : 0
QOSDID : 1
PECI : 192
DII : Reserved (55)
TL : 0
CAP : 147456
ADUPTR : 00000018
ESN : 00000001
DGID : 0
NDPL : 0
DBM :
...
$
- VDID: Vritual Device ID
- ST: State Transition
- SBS: Super Block State
- SBID: Super Block ID
- PID: Placement ID
- QOSDID: QoS Domain ID
- PECI: Program/Erase Count Index
- DII: Data Integrity Index
- TL: Time Left
- CAP: Capacity
- ADUPTR: ADU Pointer
- ESN: Erase Serial Number
- DGID: Die Group ID
- NDPL: Number of Defective Planes
- DBM: Defect Bitmap
Superblock List
$ nvme kioxia sb-list /dev/nvme0 -q1 -l 0 -s 40 -o0
SB-1, PECI = 1, DII = 4
...
$
- SB: Super Block ID
- PECI: Program/Erase Count Index
- DII: Data Integrity Index
Superblock management
Operation:
- 0: Erase
- 1: Close
- 2: Free
- 3: Patrol
- 4: Flush
nvme kioxia sb-manage -q 1 -s 10 -o 1 /dev/nvme0
User Address List
$ nvme kioxia ua-list /dev/nvme0 -q1 -s 10 -l4096 -o0
00000000: UA-00000000, METADATA = 00000001 00000002
00000001: UA-00000003, METADATA = ffffffffffffffff ffffffffffffffff
...
$
Address Change Order
Note: If there are no ACO’s, the output will be blank.
nvme kioxia aco-list /dev/nvme0 -q1 -s0 -l4096 -o0
Write
Prerequisite: Data file that is the size of <# of ADUs> * <ADU Size> Note: An open Super
Block must be closed in order to delete a QoS Domain.
cd <path to SEF_SDK>/lib
nvme kioxia write -a 4 -n 1 -f -D
devtool/nvme-cli-sef/sefWriteData_16K.txt /dev/nvme0
OPENSB=$(nvme kioxia sb-list -q1 -l3 -s8 -o0 /dev/nvme0 | cut -d "-"
-f2 | cut -d "," -f1)
nvme kioxia sb-manage -q 1 -s $OPENSB -o 1 /dev/nvme0
Read
Prerequisite: Issue a write command Note: A read command requires the source flash address
cd <path to SEF_SDK>/lib
nvme kioxia write /dev/nvme0 -a 4 -n 1 -f -D
devtool/nvme-cli-sef/sefWriteData_16K.txt > WRITE.log
SUA=$(cat WRITE.log | grep "SFLA:" | cut -d ":" -f2 | xargs)
SUA=0x$SUA
OPENSB=$(nvme kioxia sb-list -q1 -l3 -s8 -o0 /dev/nvme0 | cut -d "-"
-f2 | cut -d "," -f1)
nvme kioxia sb-manage -q 1 -s $OPENSB -o 1 /dev/nvme0
nvme kioxia read /dev/nvme0 -f $SUA -a 4 -d sefReadData_16K.txt -u 0
Copy
Prerequisite: Issue a write command Note: A copy command requires the source flash address The
flash address 0xffffffff indicates automatic allocation.
cd <path to SEF_SDK>/lib
nvme kioxia write /dev/nvme0 -a 4 -n 1 -f -D
devtool/nvme-cli-sef/sefWriteData_16K.txt > WRITE.log
SUA=$(cat WRITE.log | grep "SFLA:" | cut -d ’:’ -f2 | xargs)
SUA=0x$SUA
OPENSB=$(nvme kioxia sb-list -q1 -l3 -s8 -o0 /dev/nvme0 | cut -d ’-’
-f2 | cut -d ’,’ -f1)
nvme kioxia sb-manage -q 1 -s $OPENSB -o 1 /dev/nvme0
nvme kioxia sb-manage -q 1 -s 0xffffffff -o 0 /dev/nvme0
OPENSB=$(nvme kioxia sb-list -q1 -l3 -s16 -o0 /dev/nvme0 | cut -d ’-’
-f2 | cut -d ’,’ -f1)
IFS="
"
OPENSBS=( $OPENSB )
echo open superblocks 1:${OPENSBS[0]} 2:${OPENSBS[1]}
devtool/script_for_cli/gen_copy_bitmap_bin.sh -a 4 -b 0 -c 0 -d 0 -e
$SUA -z COPY.out
nvme kioxia copy /dev/nvme0 -q 1 -s ${OPENSBS[1]} -n 256 -b
devtool/script_for_cli/data/COPY.out -B
devtool/nvme-cli-sef/sefNlcOBitmap.bin -A
devtool/nvme-cli-sef/sefNlcACR.bin -R 0xffffffff
Get Log
cd <path to SEF_SDK>/lib
nvme kioxia write /dev/nvme0 -a 4 -n 1 -f -D
devtool/nvme-cli-sef/sefWriteData_16K.txt
nvme kioxia get-log /dev/nvme0 -n 1 -i 0xd5 -l1024 -o0 -s0 -B0 -a0
Asynchronous event change request
Prerequisite: An event must be triggered
Usage: nvme kioxia aen-rq <device> [OPTIONS]
Wait on async event notification request
Options:
[ --get-log, -l ] --- read the log after getting the event
[ --del-log, -d ] --- delete the log entry after reading
[ --num-events=<NUM>, -n <NUM> ] --- number of events to wait for
[ --timeout=<NUM>, -t <NUM> ] --- NVME command timeout in ms