Agent Remediation

Issue: Scheduler Not Running

Windows

To test Windows Scheduler, run this command:

CODE

Get-ScheduledTask -taskname SUPERVISOR_TASKS

Output Examples:

Task is enabled. This is a good state.

CODE

C:\Users\Administrator> Get-ScheduledTask -taskname SUPERVISOR_TASKS

    TaskPath                                       TaskName                          State
    --------                                       --------                          -----
    \Armor Defense\                                SUPERVISOR_TASKS                  Ready

Task is disabled. This is a bad state.

CODE

C:\Users\Administrator> Get-ScheduledTask -taskname SUPERVISOR_TASKS

    TaskPath                                       TaskName                          State
    --------                                       --------                          -----
    \Armor Defense\                                SUPERVISOR_TASKS                  Disabled

Task is missing. This is a bad state.

CODE

C:\Users\Administrator> Get-ScheduledTask -taskname SUPERVISOR_TASKS
    Get-ScheduledTask : No MSFT_ScheduledTask objects found with property 'TaskName' equal to 'SUPERVISOR_TASKS'.  Verify
    the value of the property and retry.
    At line:1 char:1
    + Get-ScheduledTask -taskname SUPERVISOR_TASKS
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : ObjectNotFound: (SUPERVISOR_TASKS:String) [Get-ScheduledTask], CimJobException
        + FullyQualifiedErrorId : CmdletizationQuery_NotFound_TaskName,Get-ScheduledTask

For bad states, please run this script:

CODE

    $start = "00:{0}" -f [datetime]::Now.AddMinutes((3 + 15)).Minute.ToString("00");
    $interval = 15;
    $schedule = "MINUTE";
    $user = "NT AUTHORITY\SYSTEM";
    $taskName = "\Armor Defense\SUPERVISOR_TASKS";
    $taskRun = "c:\.armor\opt\armor-supervisor.exe get-tasks";
    $arguments = "/create /f /sc `"${schedule}`" /tn `"${taskName}`" /tr `"${taskRun}`" /np /st `"${start}`" /mo `"$interval`" /k /ru `"${user}`"";
    Start-Process -FilePath "schtasks.exe" -ArgumentList $arguments

Example:

CODE

C:\Users\Administrator> $start = "00:{0}" -f [datetime]::Now.AddMinutes((3 + 15)).Minute.ToString("00");
C:\Users\Administrator> $interval = 15;
C:\Users\Administrator> $schedule = "MINUTE";
C:\Users\Administrator> $user = "NT AUTHORITY\SYSTEM";
C:\Users\Administrator> $taskName = "\Armor Defense\SUPERVISOR_TASKS";
C:\Users\Administrator> $taskRun = "c:\.armor\opt\armor-supervisor.exe get-tasks";
C:\Users\Administrator> $arguments = "/create /f /sc `"${schedule}`" /tn `"${taskName}`" /tr `"${taskRun}`" /np /st `"${start}`" /mo `"$interval`" /k /ru `"${user}`"";
C:\Users\Administrator> Start-Process -FilePath "schtasks.exe" -ArgumentList $arguments
C:\Users\Administrator> Get-ScheduledTask -taskname SUPERVISOR_TASKS

    TaskPath                                       TaskName                          State
    --------                                       --------                          -----
    \Armor Defense\                                SUPERVISOR_TASKS                  Ready

Linux:

Verify job exists in /etc/cron.d/armor-job-SUPERVISOR_TASKS

If not, run this script:

CODE

SEED=$(( $RANDOM % 14 ))
CRON_EXPRESSION="${SEED},$((${SEED} + 15)),$((${SEED} + 30)),$((${SEED} + 45))"
CRON_FILE=/etc/cron.d/armor-job-SUPERVISOR_TASKS
echo -e "${CRON_EXPRESSION} * * * *\troot\t/opt/armor/armor-supervisor get-tasks" > ${CRON_FILE}

Output Examples:

Task is enabled. This is a good state.

CODE

[root@myhost ~]# cat /etc/cron.d/armor-job-SUPERVISOR_TASKS
0,15,30,45 * * * * root /opt/armor/armor-supervisor get-tasks

Task is missing. This is a bad state.

CODE

[root@myhost ~]# cat /etc/cron.d/armor-job-SUPERVISOR_TASKS
cat: /etc/cron.d/armor-job-SUPERVISOR_TASKS: No such file or directory
[root@myhost ~]#

Script. This will re-add the cron job.

CODE

[root@myhost ~]# SEED=$(( $RANDOM % 14 ))
[root@myhost ~]# CRON_EXPRESSION="${SEED},$((${SEED} + 15)),$((${SEED} + 30)),$((${SEED} + 45))"
[root@myhost ~]# CRON_FILE=/etc/cron.d/armor-job-SUPERVISOR_TASKS
[root@myhost ~]# echo -e "${CRON_EXPRESSION} * * * *\troot\t/opt/armor/armor-supervisor get-tasks" > ${CRON_FILE}
[root@myhost ~]# cat /etc/cron.d/armor-job-SUPERVISOR_TASKS
8,23,38,53 * * * * root /opt/armor/armor-supervisor get-tasks

Grep cron log. This is a good state.

CODE

[root@myhost ~]# grep "FAILED to authorize user with PAM (Authentication token is no longer valid; new one required)" /var/log/cron

Grep cron log. This is a bad state. See commands for unexpiring.

CODE

[root@myhost ~]# grep "FAILED to authorize user with PAM (Authentication token is no longer valid; new one required)" /var/log/cron
Aug 12 23:00:00 myhost crond[9594]: (root) FAILED to authorize user with PAM (Authentication token is no longer valid; new one required)

Commands to check expired password. Good state.

CODE

[root@myhost ~]# chage -l root
Last password change : Aug 17, 2020
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7

Commands to check expired password. Bad state. Reset password.

CODE

[root@myhost ~]# chage -l root
Last password change : password must be changed
Password expires : password must be changed
Password inactive : password must be changed
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7

Password reset

CODE

[root@myhost ~]# passwd root
Changing password for user root.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

Connectivity/Firewall

Ensure that your firewalls are opened to the hosts as noted in Firewall Rules section of the Pre-Installation guide.

Windows Test Connectivity

IP address and port for these commands will need to be entered and are found in the Firewall Rules section of the Pre-Installation guide.

CODE

(New-Object System.Net.Sockets.TCPClient).BeginConnect("IP_ADDRESS",PORT,$null,$null).AsyncWaitHandle.WaitOne(1000,$false);

Windows connectivity test output should return a value of "true."
Windows connectivity test should not return a value of "false."
- If test returns "false," investigate firewall blockages.

Examples:

This is the command run and value returned when there is connectivity to the service.

CODE

C:\Users\Administrator> (New-Object System.Net.Sockets.TCPClient).BeginConnect("8.8.8.8",443,$null,$null).AsyncWaitHandle.WaitOne(1000,$false);
True

This is the command run and value returned when there is no connectivity to the service and your firewalls need to be checked for blockages.

CODE

C:\Users\Administrator> (New-Object System.Net.Sockets.TCPClient).BeginConnect("8.8.8.8",442,$null,$null).AsyncWaitHandle.WaitOne(1000,$false);
False

Linux Test Connectivity

IP address and port for these commands will need to be entered and are found in the Firewall Rules section of the Pre-Installation guide.

CODE

timeout 5 bash -c "cat < /dev/null > /dev/tcp/IP_ADDRESS/PORT" && echo $?

Linux connectivity test output should return a value of "0."
- If test returns anything other than "0," investigate firewall blockages.

Examples:

This is the command run and value returned when there is connectivity to the service.

CODE

root@myhost:~# timeout 5 bash -c "cat < /dev/null > /dev/tcp/8.8.8.8/443" && echo $?
0

This is the command run and value returned when there is no connectivity to the service and your firewalls need to be checked for blockages.

CODE

root@myhost:~# timeout 5 bash -c "cat < /dev/null > /dev/tcp/8.8.8.8/442" && echo $?
124

Powershell Version

Following Microsoft's documentation, upgrade to at least PowerShell version 5.

You can get PowerShell version 5 from the Microsoft.

No TLS 1.2

You can follow this Microsoft KB in order to enable TLS 1.2.

Having trouble with your upgrade? Here are some solutions to common issues.