SilverCreek Release Notes


Release: 2022 beta1 Build 220208 Date: 2022-02-08

New SNMP over DTLS Support (RFC 6353)

SilverCeek now supports SNMPv3 over Datagram Transport Layer Security (DTLS) Transport Model for SNMP (RFC6353). Customers can now connect to SNMP agents that implement SNMP over DTLS and run SilverCreek automatic SNMPv3 tests to test their SNMP over DTLS and MIB implementations.

SilverCreek Notification monitor now supports receiving notifications over DTLS. Customers can also test SNMP traps and informs sent over DTLS/UDP secure transport now.

The new SNMP over DTLS is based on the following RFCs:

  • RFC 6353 Transport Layer Security Transport Model for SNMP (DTLS /UDP)

  • RFC 5343 SNMP Context EngineID Discovery

  • RFC5591 Transport Security Model for the (TSM) SNMP.

SNMP Notification Monitor Enhancement

Added a new configuration field "SNMPv3 agent port" so the SNMP engine time synchronizing requests can be sent to a custom port. It was hard-coded to always use the default port 161.

Bug fix:

SNMP V3 trap decoding could get duplicated elements in the resulted lists when searching applicable context procedure to decode. The trapData passed to callback may contain duplicated "version", "Message_global_header", and "Message_secu_parameters" elements.

Added engineTime and engineBoots value in Message_secu_parameters to be delivered to trap callbacks

Bug fix

Test11.1.13 was generating an error " expr: bareword error" and an undefined exptStatus variable error.

Bug fix

SilverCreek BSCM2 (v1v2c only) edition was generating errors when switching to SNMPv3 Diffie-hellman page. The trap monitor is also generating errors when toggling various SNMPv3 buttons states.


 

Release: 2020 Build 20.02.10 beta

Date: 2020-02-10

SilverCreek now provides a Python API to perform SNMP operations, receiving traps, and automate executing tests in command line. Both Python 2.7.x and 3.x are supported. See samples/CLI-Python for sample scripts.

Python command reference can be found in SilverCreek developer guide.

Added a new command to provide a way to rerun given type of tests (e.g. Failed tests) automatically after the initial test run, for a desired number of times

This API supports excluding a set of test-suites and tests from reruns.

It supports saving test results of these reruns.

In Tcl Command Line API, it is snmptcl::reruntests

In Python Command Line API, it is RunCLI_Rerun_Tests.

Tests such as test1.1.4, 1.1.5, 1.1.9.2, and test1.1.11.17, test2.1.3.1, 2.1.3.2, 2.2.4.1, 2.2.4.2, 2.3.1.4 and test2.4.1.2, 3.1.3.1, 3.1.3.2, 3.2.4.1, 3.2.4.2, 3.3.1.4 and test3.4.1.2 were sending duplicated requests that should have been suppressed.

The command snmptcl::getDetails was returning the test log file names incorrectly. Changed to retrieve and return the last log file content.

Changed to save the fine-tune testing configuration for "Ignore objects in read-write testing" into agent setup (context) file so that different DUT can use different set of objects to ignore.

Bug Fix: Saving test results (test6.5.3.3) in test-suite 6.0 into HTML format was generating errors if the DUT sends back a response contains non-printable characters. The test description of test2.3.2.4 and test3.3.2.4 should say a noSuchInstance exception is expected, rather than noSuchName.

Test description about the operations performed in test1.1.9.2, 2.3.1.4 and 3.3.1.4 were incorrect.

Clicking the 'Stop' button has no effect when running test1.1.11.18, 2.4.1.4 and 3.4.1.4.

The option -expectexcept in snmptcl::snmpcmd was not working properly. It may trigger unexpected WARNING in the test 2.3.2.4 etc.

Test1.1.16.4, 2.3.2.4 and 3.3.2.4 were possibly sending a GET request on a non-existent object and expecting the DUT to return noSuchInstance exception.

The boundary tests such as test1.1.11.3, 2.5.1.2 and 5.1.2 were not including the failed OID and value in the failure message and were aborting unexpectedly upon seeing the first error.

Changed the procedure rsp2text to display both the resulted OID and its interpreted form, For example: 3.6.1.2.1.4.2.0(IP-MIB:ipDefaultTTL.0 ).

Test1.1.11.1, 1.1.11.1.1, 2.4.1, 2.4.1.3, 3.4.1 and 4.1.3 should allow "badValue" or "inconsistentValue" as possible return errorStatus when setting arbitrary variables with open semantics. Many columnar objects may be defined to return inconsistentValue even when setting it to a valid value.

Test1.1.11.1.1 and test3.4.1.3 could generate errors due to an uninitialized variable, and they should NOT allow timeout.

Test2.1.2.3 and test3.1.2.3 may report false failures on dynamic (ephemeral) columnar variables.

Updated command snmptcl::oidcmp to support comparing an OID with an empty string. This change will cause tests such as test3.2.5.2 and 2.2.5.2 to report details on failed variable bindings that don't contain an OID.

Release: 2018 Build 18.04.29

Date: 2018-04-29

New Tests: Added 14 new tests in test-suite 4.0 USM MIB test-suite to check if the DUT validates UTF-8 encoded SNMPv3 user name properly when creating new users.

New Tests: Added 18 new tests in test-suite 6.0 SNMP apps test-suite to check if the DUT validates UTF-8 encoded snmpTargetAddrTag and snmpTargetAddrTagList value properly.

New Tests: Added 19 new tests in test-suite 9.0 SNMP community MIB test-suite to check if the DUT validates UTF-8 encoded snmpCommunitySecurityName and snmpCommunityContextName value properly.

New Tests: Added test1.1.11.1.2, test2.4.1.5 and test3.4.1.5 to check setting read-write objects with duplicated variable-bindings.

New Test: Added test6.4.3.3.1 to check if the DUT allow setting snmpTargetAddrTagList to an empty value.

New Feature: Added a new feature "Disable Get-bulk tests in Protocol test-suite and standard test-suites"

Improvement: Changed the error "Agent failed more than x times." to a clearer message: "Agent failed more than n times. Failed: x" in all affected tests.

Bug Fix: Tests such as test4.20.8, 4.20.10, 4.14.3, 4.17.3, 4.10.1.7, 4.14.4, 4.10.1.4, etc. in test-suite 4.0 (USM-MIB tests) were not supporting SHA2 and were all reporting 'Unknown authentication protocol SHA-224,SHA-256,SHA-384, and SHA-512'.

Bug Fix: The test packet sent in the test3.5.1.4 did notcontain a message authentication digest encoded properly, and was reporting a false failure caused by a “usmStatsWrongDigests” report PDU.

Bug Fix: A severe performance degradation was triggered when using the upper level commands such as snmptcl::snmpget, snmptcl::snmpnext etc. when there are many Agent Capability statements loaded.

Bug Fix: The options such as snmptest(errors), snmptest(skipSetTests) etc. set in the SilverCreek resource file (.silvercreekrc) were cleared and reset to default unexpectedly when running tests in CLI.

Bug Fix: Inform response tests was using snmptest(instance) parameter incorrectly and it should have used only the value 0.

Bug Fix: The advanced parameter "BulkWalkOids" was not set in CLI properly and was sometimes causing errors when running tests from CLI.

Bug Fix: The function “Register_add_for_inform” was configuring agent context using the values in the global snmpcontext array mistakenly.

Bug Fix: Test1.1.11.1.1, test2.4.3.1 and test3.4.3.1 were accepting badValue, inconsistentValue, noAccess as acceptable return error status incorrectly.

Bug Fix: Test9.1.7.4 and 9.1.7.5 were showing PASSED even if there are not suitable rows found to test

Bug Fix: A set of tests in test-suite 9.0 (SNMP Community MIB)were expecting SNMPv2c error status even in SNMPv1 mode

Improvement: Changed test6.4.1.x, 6.4.3.x and 6.4.8.x to not require user interaction to configure a valid local IP address since any IPv4 address will work.

Bug Fix: Tests such as test20.0.1, test10.1.3 that call mib2_find_ifIndex to determine the interface in use were not able to abort when instructed.

Bug Fix: Test1.1.2.1, 2.1.2.2, 3.1.2.2 were not stopping even if the total errors are equal to or more than the set maximum allowed failure count.

Bug Fix: snmptcl::rowStatusOp and snmptest::rowStatusOp commands were deleting row even if the option -autodestroy is set to false (0).

Bug Fix: Test3.8.1 was not skipping SET tests even if the option 'Test | Disable SET Tests in Protocol Test-suites' is enabled

Bug Fix: Tests such as test20.0.1, test10.1.3 that call mib2_find_ifIndex to determine the interface in use were not able to abort when instructed.

Release: 2017 Build 17.01.22

Date:2017-01-22

New Feature: SilverCreek now fully supports SHA224, SHA256, SHA384 and SHA512 for SNMPv3 (RFC 7860).

New Tests: Added four new tests: test4.26.13, test4.26.14, test4.26.15 and test4.26.16 to check authentication bypass attack on SHA-2.

New Feature: Added a new option "Test | Abort Tests that Require User Interaction" to abort interactive tests.

New Test: Added a new test4.21.3 to check if the DUT can generate unSupportedSecurityLevel report.

New Tests: Added three new tests test1.1.11.5.0, test2.4.3.2.0 and test3.4.3.2.0 to test setting variables using all valid values in the defined range.

New Feature: Added support to mark SNMPv1 and SNMPv2c traps received as "problematic" and also display them in red color if the community string received is different than the configured one.

Critical Fix: SilverCreek was sending SNMPV2c inform response with a hard-coded community string 'public'.

Critical Fix: SilverCreek trap receiver was sometimes sending out an unSupportedSecurityLevel report unexpectedly.

Enhancement] SilverCreek command "snmptcl::context::create" now has a new option "-trapoctetstringformat" to control how to parse Octet-String variable in SNMP traps.

Enhancement: Make certain tests to return 'Untested' rather than 'Error' when Python and Scappy that they dependent on haven't been installed.

Enhancement: Test1.1.11.6.0, 2.4.3.3.0 and 3.4.3.3.0 now set using all valid enumerated values.

Enhancement: Issue a follow-up get request in test3.4.3.x to make sure the DUT didn't actually assign invalid values to the variables tested.

Bug Fix: Trying to configure trap options in In SilverCreek BSCM1's trap monitor was getting an error complaining missing scmain::show_Icon procedure.

Critical Fix: SilverCreek trap monitor was not able to display incoming traps if there isn't already a silvercreek resource file saved in a previous run of SilverCreek.

Change: Updated SilverCreek AES192 and AES256 to work with a couple of widely deployed agent implementations, although 3DES, AES192 and AES256 are never standardized by IETF.

Bug Fix: Copying and pasting texts into SilverCreek console was not working when there is a test is currently selected or highlighted.

Bug Fix: Test4.21.1 and test4.21.2 were not checking snmpInPks and usmStatsUnsupportedSecLevel counters properly.

Bug Fix: Remove repetitive entries of snmptcontext(authkey) and snmptest(privkey) in the saved agent context file.

Bug Fix: View test details dialog was not showing the full result of the last test run by default.

Bug Fix: Test4.19.7 should be run for any connected agent user context that is not a 'read-only' user.

Bug Fix: Test 4.15.4 generates a false failure when the agent context user configured to test is authNoPriv but the clone-from user is authPriv.

Bug Fix: Test10.4.2.4 was generating an ERROR if no ipRouteAge instance is found in the DUT.

Bug Fix: SilverCreek command snmptcl::getop, snmptcl::setop, etc. was not handling its associated mibstore procedure properly.

Bug Fix: Test3.3.1.6 and test3.4.1.6 were reporting false failure when the DUT returns noSuchObject exception and noAccess error respectively.

Release: 2017 Build 17.11.10 beta

Date: 2017-11-10

New Features and Top Bug Fixes in this Release

New Tests: Added 14 new tests (test4.9.4.x, test4.9..5.x, test4.9.6.x and test4.9.7.x) in test-suite 4.0 'USM MIB' test-suite. These tests check if the DUT handles UTF-8 encoded SNMPv3 user name properly when creating new users.

New Tests: Added 18 new tests (test6.4.3.3.x and test6.5.3.3.x) in test-suite 6.0 'SNMP Apps' test-suite. These tests check if the DUT handles UTF-8 encoded snmpTargetAddrTag and snmpTargetAddrTagList value properly.

New Tests: Added 19 new tests (test9.1.3.x, test9.1.5.x and test9.1.6.x) in test-suite 9.0 'SNMP Community MIB'. These tests check if the DUT handles UTF-8 encoded snmpCommunitySecurityName and snmpCommunityContextName value properly.

New Tests: Added 3 new tests: test1.1.11.1.2, test2.4.1.5 and test3.4.1.5 into protocol test-suites for SNMPv1, v2c and v3 respectively. These tests check if the DUT handles SNMP set requests that contain objects with duplicated variable-bindings.

New Tests: Added test6.4.3.3.1 in test-suite 4.0 to check if the DUT allow setting snmpTargetAddrTagList to an empty value.

New Tests: Added test4.9.7.1 in test-suite 6.0 to check if the DUT rejects creating entries with username set to an empty string value and without catastrophic failure.

New Feature: Added a new option "Disable Get-bulk tests in Protocol test-suite and standard test-suites" that can be used by customers to skip running GET-BULK tests if DUT does not support it.

Enhancement: Changed test6.4.1.x, 6.4.3.x, 6.4.8.x not to require user interaction to configure a valid local IP address since any IPv4 address will suffice.

Enhancement: Test20.0.1 now configures an active row in etherStatsTable that monitors the interface over which SilverCreek is talking to the agent, if there is no existing entry found in etherStatsTable.

Bug Fix: Tests such as test4.20.8, 4.20.10, 4.14.3, 4.17.3, 4.10.1.7, 4.14.4, 4.10.1.4, etc. in test-suite 4.0 'USM-MIB' tests were not supporting SHA2 and were reporting 'Unknown authentication protocol SHA-224, SHA-256,SHA-384, and SHA-512'.

Bug Fix: Test1.1.11.1.1, test2.4.3.1 and test3.4.3.1 were accepting badValue, inconsistentValue, or noAccess as acceptable return error status mistakenly.

Bug Fix: Test 9.1.7.4 and test 9.1.7.5 were showing PASSED even if there are not suitable rows found to test

Bug Fix: Some tests in test-suite 9.0 (SNMP Community MIB ) were expecting SNMPv2c error status even in SNMPv1 mode

Bug Fix: Test10.1.3 and test20.0.1 that call mib2_find_ifIndex to determine the interface in use were not able to be aborted.

Bug Fix: snmptcl::rowStatusOp and snmptest::rowStatusOp commands were deleting rows created even if the option -autodestroy is set to 0.

Bug Fix: Test3.8.1 was not skipping SET operation even if 'Test | Disable SET Tests in Protocol Test-suites' is enabled