Windows

Attacks can sometimes fail to notify the Gremlin Control Plane when its connection is impacted by the attack itself. The Gremlin agent now tolerates these failures more often and attempts to resend failed notifications. This fixes attacks that end up in the HaltFaled stage that would otherwise finish in the Successful stage.

Improved the output of the Gremlin Agent validation routine that happens on startup. When validation fails, details about the failure are written to daemon.log.

Fixed an issue where attacks were incorrectly labeled HaltFailed when Gremlin fails to notify api.gremlin.com during teardown of the network impact.

For users running Gremlin on AWS, more error information is printed to the log file when AWS metadata cannot be retrieved.

Updated dependencies

Errors related to spawning subprocesses now have more detailed information useful for troubleshooting.

For hostnames supplied to network attacks, Gremlin delegates DNS queries to the operating system. When this query fails, Gremlin now attemps to resolve the name completely within the running process in an attempt to overcome operating system failures. This allows Gremlin network attacks to continue in the face of failed DNS processing.

Fixed a bug where Latency attacks would attempt to target multiple network interfaces, which is not yet supported. An error is now returned if --device is used to specify multiple network interfaces for a Latency attack. If no --device is specified, Gremlin will choose the highest priority device it finds.

Updated dependencies.

Gremlin now tears down the TCP connection pool with api.gremlin.com after successive timeout failures.

Gremlin includes the name of the targeted network interface in execution log events related to applying network impact.

Updated dependencies.

Added support for tag values to be any simple YAML datatype (boolean, integer, float, string). Previously only strings were supported.

Updated dependencies.

Fixed an issue that prevented Gremlin from ingesting Azure Tags.

Fixed an issue that made Gremlin validation unreliable.

Updated dependencies.

Gremlin's version command now prints more build information.

Updated dependencies.

Multiple network interface attacks are now supported. Details are available in Network device selection.

Updated dependencies.

Fix a bug in collect_certs when the target dropped the network connection before completing the TLS setup.

Only consider IPv4 addresses for the default Gremlin Identifier.

Updated help URLs.

Updated dependencies.

Fix a bug that prevented collect_certs from working when run against a container.

Updated dependencies.

Add a short argument (-n) for the not_less_than option.

Updated dependencies.

Introduce Certificate Expiry test for Reliability Management.

Updated dependencies.

Fixed a bug where Gremlin would not properly launch attacks that resolve to a large amount of IP addresses / blocks.

Fixed a bug where Gremlin would not properly include swap in free memory calculations, leading to incorrect attack results.

Updated dependencies.

Fixed a bug where Gremlin would attempt to allocate more memory than was available when the cgroup attribute memory.limit_in_bytes was higher than available system memory.

Updated dependencies.

Gremlin's Memory attack now has a new argument: --allocation-strategy (-s), which informs Gremlin on how to interpret other memory consumption arguments: --percent, --mb, and --gb. See more at Memory: Options

Updated dependencies.

Correctly handle proxy usernames and passwords that contain special characters. Special characters must be percent-escaped. For example, %5C is used in place of a backslash. Details are available here.

Updated dependencies.

Disk attack improvements. The Disk attack is much faster, more accurate, and safer.

Gremlin now reads custom Azure tags associated with the machine and makes those tags available for targeting.

Fixed a bug where Gremlin would fail validation if the DNS lookup of api.gremlin.com failed. This was likely to be a problem in high security environments.

Fixed a bug where Gremlin would crash on some operating systems when process collection was enabled. Gremlin avoids crashing and disables process collection when errors are detected.

Fixed a bug where Gremlin used to skip tag configuration even when it had a valid session. Gremlin now always configures tags on startup as long as it can communicate with the Gremlin control plane.

Some automatic Azure tags were not being correctly read. The azEnvironment, location, name, osType, privateIpAddress, publicIpAddress, sku, vmId, vmSetScaleName, and zone tags are all automatically read and available for targeting.

Removed NTP timestamp from gremlin check os.

Updated dependencies.

Updated Rust version used to build agent in response to CVE-2022-21658

On startup, the Gremlin agent now performs some validation on its ability to run a CPU and Latency attack. Validation results are accessible through the Clients API.

Updated dependencies

Updated dependencies

The Disk attack has been significantly improved. In most cases it is much faster, more accurate, and safer. It also uses significantly less CPU and RAM when filling disk volumes. The improved version is used when the environment variable GREMLIN_DEUCHAINN_EN1023 is set to true; all other values are treated as false. This environment variable may be ignored or removed in a future version without notice.

Fixed bug with Gremlin's IO attack cleanup when --mode r or --mode w was used. Previously, Gremlin would try to tear down files that did not exist, leading to attack failures.

Improve messages reported by the Gremlin IO attack, when file-creation errors occur.

Changed the way PUSH_METRICS boolean configuration variable is evaluated. Previously, any non-empty value other than "0" would evaluate to true (e.g. PUSH_METRICS=false would evaluate to true). This has been changed to provide expected outcomes: the only values that evaluate true are now "1", "true", and "TRUE", leaving all other values to evaluate to false.

Updated dependencies

Fixed a bug where the Gremlin agent does not properly roll back time travel attacks with an offset of 5 seconds or less.

The percent argument for Disk attacks now accepts real numbers. For example, --percent 27.5 was previously unsupported.

Gremlin now correctly determines the local hostname making the automatic local-hostname available for targetting.

API interactions made by the Gremlin agent now always send the appropriate Content-Type header value.

Updated dependencies

Fixed a bug where the CPU attack would not affect all processors on systems with more than 64 processors.

Updated dependencies

Updated dependencies

This update fixes Memory attack bugs. Previously, the amount of memory consumed would be limited to RAM. Memory attacks with this update include swap space / all virtual memory.

The Memory attack is more "aggressive" in the sense that the memory allocated by Gremlin during the attack is more difficult to swap to disk.

Updated dependencies.

The Gremlin CLI now has a gremlin check daemon subcommand which reports on the status of any running Gremlin agent.

Updated dependencies.

Updated dependencies.

Updated dependencies.

While never observed, according to the Windows API documentation, getting the current username can fail. If that happened the Gremlin Client would fail to run an attack. Instead, this version resorts to using "unknown" if the username cannot be determined.

Daemon log file management improvements. Previously, the log file was truncated at midnight. That made troubleshooting difficult. The log file is now rolled when it reaches approximately 1 MiB. Ten compressed log files are kept. With this update the current log file typically captures several days and the compressed log files typically capture a few weeks at a modest cost of approximately 2 MiB of disk space.

Fix a bug in Gremlin's argument parsing for the hostnames and ipaddresses arguments for network attacks.

Improve command-line argument parsing by providing better error messages and catching more edge cases related to illegal inputs.

Patch a vulnerability in a 3rd party library that posed a variety of memory corruption scenarios, most likely use-after-free.

Improve error messages among network attacks when an invalid network device is supplied. Error message now includes all valid devices.

Drop invalid targeting tags with a warning.

The daemon version is included in the gremlin check report.

Occasionally the Docker version was incorrectly parsed which would result in the classic driver being used for container attacks.

Some agent API traffic is now consolidated into fewer requests.

Some agent API traffic is now gzip-compressed, reducing network overhead on machines where Gremlin is installed.

Patch a vulnerability in a 3rd party library that posed a potential buffer overflow scenario

Patch a vulnerability in a 3rd party library that posed a potential scenario to operate on dangling memory references

You can now specify the SSL_CERT_FILE variable via the config.yml file. See the advanced configuration page for details on how to use it.

Gremlin now properly interprets escaped newline characters \n for values of the GREMLIN_SSL_CERT environment variable.

Gremlin now reports container and process data at a slower rate, down from every 5 seconds during active attacks (and every 10 seconds otherwise) to every 30 seconds. We've found that this data changes much less frequently than is justified for a 5-10 second interval. This should result in significantly reduced network overhead required to run Gremlin.

Updated dependencies

The Gremlin agent now writes a message to daemon.log when attacks finish. This provides observers of this log with an approximation on when attacks have ended.

Updated dependencies

Gremlin correctly reports Windows 2019 (and later)

Gremlin can be removed if the kernel driver is stopped or removed by hand

Eliminate a nuisance warning that was output at the end of an attack

Updated dependencies

Updated dependencies

Fixed a bug that prevented proper installation on Windows Server 2016

AWS Availability Zone ID (azid) is available for targeting.

AWS tags are now available for targeting.

Fix a regression introduced in 1.0.10 that prevented proper installation on Windows 8.1 / Server 2012 R2

Latency attacks are available

Updated dependencies

On Windows machines with many cores, CPU attack was not utilizing as much CPU time as expected.

Improve error messaging when Gremlin fails to find an IP address for a hostname supplied with the --hostname argument, which can be passed to any network attack. Error message now mentions failures due to specifying a hostname that maps to an invalid DNS record type, such as NS.

Patch a vulnerability in a 3rd party library that posed a potential denial of service to Gremlin's outbound https connections. In practice this is 100% mitigated unless connecting Gremlin through a malicious SSL proxy

Updated dependencies

Add DNS attack

Added more detail to error messages that occur when Gremlin fails to do a DNS lookup of a hostname. Previously the error message did not include the reason for the lookup failure. An example of the new detail we've added is: failed to lookup address information: Name does not resolve.

Fixed a bug where the Gremlin Disk attack would not clean up the impact files it created if it was halted from the UI.

Added additional validity checks on data returned from the Windows API. This improves error handling when the target system does not behave as Gremlin expected.

We now collect an approximate host boot time, this will aid Gremlin to better recognize unique hosts on your team.

Select a default network interface in more cases (also used when Gremlin identifier isn't specified).

Initial release of Gremlin for Windows, allowing attacks to be run on Windows environments. See more about running Gremlin on Windows.