Prior to the AXOS-R22.3, there were only 2 alarms related to "disk-space-tca":
- WARNING (Raised at 80% threshold - Alarm clears at 75% threshold)
- CRITICAL (Raised at 90% threshold - Alarm changes to WARNING at 85% threshold)
In AXOS-R22.3 and above, the "disk-space-tca" alarm is raised with the following severities:
- MINOR (Raised at 50% threshold - Alarm clears at 48% threshold)
- WARNING (Raised at 80% threshold - Alarm changes to MINOR at 75% threshold)
- CRITICAL (Raised at 90% threshold - Alarm changes to WARNING at 85% threshold)
Note: In AXOS-R22.4, the WARNING alarm will be changed to a MAJOR alarm. Here are some questions that BSPs may have about this alarm: - How do I verify what is the current FLASH space utilization on my cards?
BSPs can run the "
show info" command to look specifically for the "
/media/sda2" to confirm the current FLASH space utilization.
Example:
AXOS-E7-2-GPON# show info
info 1/1
uptime "1 day 19:09 (hh:min)"
cpu-usage 14 %
ram-usage 46 %
available-memory 2129976 Kb
intake-temp 38.2 degC
fan-speed 45 %
DISK VOLUME USED
---------------------
/media/sda1 47 %
/media/sda2 50 % <================= Indicates FLASH space
/mnt/rw 37 %
/var/volatile 13 %
info 1/2
uptime "1 day 19:09 (hh:min)"
cpu-usage 18 %
ram-usage 44 %
available-memory 2214776 Kb
intake-temp 41.3 degC
DISK VOLUME USED
---------------------
/media/sda1 47 %
/media/sda2 48 % <================= Indicates FLASH space
/mnt/rw 37 %
/var/volatile 13 %
AXOS-E7-2-GPON#
- I upgraded my AXOS system and I now have a "disk-space-tca" alarm on one (or more) cards. How do I clear this alarm?
If you recently upgraded your E7 or E3 to AXOS-R22.3 or higher and are seeing "disk-space-tca" alarm on one or more cards, use below steps will help them clear the alarm as far as possible using the CLI.
Step 1: Lets take an example here where we have a dual-card system which is running AXOS-R22.3 and one of the AXOS card has a disk-space-tca alarm as shown below:
E7-AXOS# show alarm active
alarm active
total-count 1
alarm 1
ne-event-time 2022-10-06T04:34:53-05:00 id 2612 name disk-space-tca instance-id 9.1 perceived-severity MINOR category TCA address /config/system/cli_conf/slot[location='1/1'] primary-element media-sda2
E7-AXOS#
Step 2:Verify the FLASH utilization using "show info" command:
E7-AXOS# show info
info 1/1
uptime "5 days 8:38 (hh:min)"
cpu-usage 14 %
ram-usage 41 %
available-memory 2312148 Kb
intake-temp 27.4 degC
fan-speed 44 %
DISK VOLUME USED
---------------------
/media/sda1 47 %
/media/sda2 53 % <================ We have disk-space-tca on card 1/1 since utilization is above 50%
/mnt/rw 37 %
/var/volatile 13 %
info 1/2
uptime "5 days 8:38 (hh:min)"
cpu-usage 17 %
ram-usage 41 %
available-memory 2311972 Kb
intake-temp 27.4 degC
DISK VOLUME USED
---------------------
/media/sda1 47 %
/media/sda2 49 %
/mnt/rw 37 %
/var/volatile 13 %
E7-AXOS#
Step 3: Run "
show file contents maxsize" command to identify top 10 files which are largest in size from amongst following folders: techlog/core/loghistory/bar/certificate/config/ont-debug/ont-sip-config/ont-snapshot/tcpdump
Note: This command is only available in R22.3 for E7 and E3 nodes. Support for the ASM and CLX systems will come in future releases.E7-AXOS# show file contents maxsize
file 1/1
contents
maxsize
FOLDERNAME CARD FILENAME SIZE LAST MODIFICATION TIME
-------------------------------------------------------------------------------------------------------------------------------------------------
techlog 1/1 techlog-E7-AXOS-01-02-072006966737-2021_07_29T15_26_05-common.tar.gz 105581315 Thu Jul 29 15:26:35 2021
loghistory 1/1 boot-000006.tar.gz 10982920 Wed Sep 1 03:33:44 2021
loghistory 1/1 boot-000007.tar.xz 10693360 Sat Aug 7 01:46:30 2021
loghistory 1/1 boot-000010.tar.xz 10081988 Fri Feb 4 15:24:47 2022
loghistory 1/1 boot-000004.tar.gz 9599022 Fri Aug 7 23:57:48 2020
loghistory 1/1 boot-000008.tar.xz 9089848 Tue Jan 18 03:34:40 2022
loghistory 1/1 boot-000009.tar.xz 2318336 Tue Apr 5 02:33:30 2022
loghistory 1/1 boot-000005.tar.gz 1210119 Thu Apr 1 21:34:08 2021
loghistory 1/1 sdkout.sav 524564 Fri Feb 7 04:30:01 2020
loghistory 1/1 sdkout 339892 Tue Apr 13 16:38:04 2021
-------------------------------------------------------------------------------------------------------------------------------------------------
number-of-files 10
total-size 160421364
file 1/2
contents
maxsize
FOLDERNAME CARD FILENAME SIZE LAST MODIFICATION TIME
-------------------------------------------------------------------------------------------------------------------------------------------------
loghistory 1/2 boot-000006.tar.xz 11154400 Sat Aug 7 01:46:17 2021
loghistory 1/2 boot-000005.tar.gz 9663734 Wed Sep 1 03:33:47 2021
loghistory 1/2 boot-000003.tar.gz 9294482 Fri Aug 7 23:57:44 2020
loghistory 1/2 boot-000007.tar.xz 8489304 Tue Jan 18 03:33:59 2022
loghistory 1/2 boot-000009.tar.xz 8218384 Fri Feb 4 15:24:24 2022
loghistory 1/2 boot-000004.tar.gz 1125709 Tue May 11 04:07:17 2021
loghistory 1/2 boot-000008.tar.xz 983040 Tue Apr 5 02:33:23 2022
loghistory 1/2 sdkout.sav 641995 Sat Aug 8 00:00:01 2020
loghistory 1/2 boot-000002.tar.gz 289710 Fri Aug 7 23:18:45 2020
config 1/2 startup-config.xml 285163 Tue Oct 11 02:05:05 2022
-------------------------------------------------------------------------------------------------------------------------------------------------
number-of-files 10
total-size 50145921
E7-AXOS#
Step 4: From above output, delete the techlog and loghistory files listed on card 1/1 to be biggest in size.
Run "
delete file techlog <slot/card> filename all" to remove ALL older techlog files on a specific card
E7-AXOS# delete file techlog 1/1 filename all
OK
E7-AXOS#
Note: Change the card number based on the card which is reporting the alarm in each case.Run following command "
delete file loghistory <slot/card> filename <filename>" to delete the loghistory files that were shown in the output of "show file contents maxsize" above for card 1/1:
E7-AXOS# delete file loghistory 1/1 filename boot-000006.tar.gz
OK
E7-AXOS# delete file loghistory 1/1 filename boot-000007.tar.xz
OK
E7-AXOS# delete file loghistory 1/1 filename boot-000010.tar.xz
OK
E7-AXOS# delete file loghistory 1/1 filename boot-000004.tar.gz
OK
E7-AXOS# delete file loghistory 1/1 filename boot-000008.tar.xz
OK
E7-AXOS# delete file loghistory 1/1 filename boot-000009.tar.xz
OK
E7-AXOS# delete file loghistory 1/1 filename boot-000005.tar.gz
OK
E7-AXOS# delete file loghistory 1/1 filename sdkout.sav
OK
E7-AXOS# delete file loghistory 1/1 filename sdkout
OK
E7-AXOS#
CAUTION: If you execute "show file contents maxsize" and see any CONFIG files in the list, DO NOT proceed to deleting any of those config files. See example below. E7-AXOS# show file contents maxsize
file 1/1
contents
maxsize
FOLDERNAME CARD FILENAME SIZE LAST MODIFICATION TIME
-------------------------------------------------------------------------------------------------------------------------------------------------
config 1/1 startup-config.xml 285163 Tue Oct 11 02:05:05 2022
config 1/1 startup-config.20221011.020505.xml 285163 Mon Oct 10 02:05:05 2022
config 1/1 startup-config.20221010.020505.xml 285163 Sun Oct 9 02:05:05 2022
config 1/1 startup-config.20221009.020505.xml 285163 Sat Oct 8 02:05:05 2022
config 1/1 startup-config.20221008.020505.xml 285163 Fri Oct 7 02:05:05 2022
config 1/1 startup-config.20221006.043000.xml 279495 Thu Oct 6 02:05:05 2022
config 1/1 startup-config.20221006.020505.xml 279495 Wed Oct 5 02:05:05 2022
config 1/1 startup-config.20221005.020505.xml 279495 Tue Oct 4 02:05:05 2022
config 1/1 startup-config.20221004.020505.xml 279495 Mon Oct 3 02:05:05 2022
config 1/1 startup-config.20221003.020505.xml 279495 Sun Oct 2 02:05:05 2022
-------------------------------------------------------------------------------------------------------------------------------------------------
number-of-files 10
total-size 2823290
file 1/2
contents
maxsize
FOLDERNAME CARD FILENAME SIZE LAST MODIFICATION TIME
-------------------------------------------------------------------------------------------------------------------------------------------------
config 1/2 startup-config.xml 285163 Tue Oct 11 02:05:05 2022
config 1/2 startup-config.20221011.020505.xml 285163 Mon Oct 10 02:05:05 2022
config 1/2 startup-config.20221010.020505.xml 285163 Sun Oct 9 02:05:05 2022
config 1/2 startup-config.20221009.020505.xml 285163 Sat Oct 8 02:05:05 2022
config 1/2 startup-config.20221008.020505.xml 285163 Fri Oct 7 02:05:05 2022
config 1/2 startup-config.20221006.043000.xml 279495 Thu Oct 6 02:05:05 2022
config 1/2 startup-config.20221006.020505.xml 279495 Wed Oct 5 02:05:05 2022
config 1/2 startup-config.20221005.020505.xml 279495 Tue Oct 4 02:05:05 2022
config 1/2 startup-config.20221004.020505.xml 279495 Mon Oct 3 02:05:05 2022
config 1/2 startup-config.20221003.020505.xml 279495 Sun Oct 2 02:05:05 2022
-------------------------------------------------------------------------------------------------------------------------------------------------
number-of-files 10
total-size 2823290
E7-AXOS#
Files which are safe to delete from the system are anything that belongs to following directories:
- TECHLOG
- CORE
- LOGHISTORY
- ONT-DEBUG
- ONT-SNAPSHOT
- TCPDUMP
Here are some command syntax to help you with deleting files from any of these folders:
CORE file:
delete file core <slot/card> filename allONT-DEBUG files
: delete file ont-debug <slot/card> filename allONT-SNAPSHOT:
delete file ont-snapshot <slot/card> filename allTCPDUMP
: delete file tcpdump <slot/card> filename all IMPORTANT NOTE: WAIT for around 5 minutes after you have performed the file deletion to see the updated FLASH info or if the alarm has cleared.
Step 5: Verify if the FLASH on the card that you are working has come below the Clear threshold level using "show info" command:
E7-AXOS# show info
info 1/1
uptime "5 days 8:47 (hh:min)"
cpu-usage 14 %
ram-usage 42 %
available-memory 2305928 Kb
intake-temp 30.0 degC
fan-speed 44 %
DISK VOLUME USED
---------------------
/media/sda1 47 %
/media/sda2 48 % <====================== Space reduced from 53% to 48%
/mnt/rw 37 %
/var/volatile 13 %
info 1/2
uptime "5 days 8:47 (hh:min)"
cpu-usage 17 %
ram-usage 41 %
available-memory 2312592 Kb
intake-temp 29.9 degC
DISK VOLUME USED
---------------------
/media/sda1 47 %
/media/sda2 49 %
/mnt/rw 37 %
/var/volatile 13 %
E7-AXOS#
Since the CLEAR threshold for a MINOR "disk-space-tca" alarm is 48%, the alarm should clear from the system.
E7-AXOS# TCA ALARM CLEAR 2612 'disk-space-tca' at 10-11-2022 13:19:53/config/system/cli_conf/slot[location='1/1'] media-sda2
E7-AXOS#
Perform
Step 2 to
Step 5 if additional log files are needed to be cleared to reduce the disk space further.
- I followed the steps listed in the above KB and I still have the alarm. What do I do next?
If after following all the above steps the disk-space-tca alarm is still present and there are no other relevant logs that you can delete, Contact TAC for further assistance.