Live feed forging formed the base of the Professor's "Paris Plan" to rescue Lisbon from police custody in Money Heist (Season 4).  A smart attacker always makes or covers up his tracks by forging something. Cyber-attacks inflict loss of reputation and wealth worth millions.

Enterprise applications generate petabytes terabytes of logs. What's the big deal if someone injects few more lines? It is like adding few drops in ocean.

Note: This post is more specific to CRLF log injection attack, used to corrupt the integrity of a log file.

Motive behind Log Forging

Let's try to find out, why the attacker will be interested in injecting false entries in the log. Followings come to mind:

  1. To cover up attacks, tracks or digital foot prints on your system.
  2. To buy time to carry on the attack. And slow down the detection process.
  3. To trick log monitoring / audit systems not to trigger alert.
  4. To initiate an attack by injecting malicious code in log. And that code gets executed when logs are opened in a vulnerable application / tool.

How Forgery Happens?

Show exactly what you want to see. Professor's team replaced the security camera feed, created a tunnel, constructed fake wall in the parking garage and replaced the prisoner. By the time investigating team realized, it was too late.

Or imagine, you have a smart home monitoring system, but thief can replace the live camera feed by pre-recorded video before entering into your premises.

Let's imagine a hypothetical vulnerable application where each successful login gets logged as info and failed login as error. If multiple failure attempts are found for a user, log monitoring system issues alert.

    
    2020-06-08 15:02:24  INFO  LoginHandler: Login successful for srccodes 
    2020-06-08 15:04:24  ERROR LoginHandler: Login failed for srccodes

Now assume a brute-force login attack is initiated on that application. This will generate lots of error messages in the log. Monitoring system will find this as suspicious activity and immediately alert us. And we'll pour cold water on attacker's hard work. 

    
    2020-06-08 15:04:14  ERROR LoginHandler: Login failed for srccodes 
    2020-06-08 15:04:25  ERROR LoginHandler: Login failed for srccodes 
    2020-06-08 15:04:38  ERROR LoginHandler: Login failed for srccodes 
    2020-06-08 15:04:53  ERROR LoginHandler: Login failed for srccodes

If attacker finds a way to inject successful login message in between failed login entries, then monitoring tool may get tricked and not issue alert immediately. And attack may continue without being detected or may get detected when it is too late.

Let me cook up one vulnerable code to explain it a bit more.

    
    	String loginId = httpServletRequest.getParameter("loginId");
		
		// Validation: space is not allowed in loginId.
		if(StringUtils.containsWhitespace(loginId)) {
			logger.info("Space not allowed in loginId " + loginId);
			:
		} else {
			:
			if(validateLogin(loginId, password)) {
				:
				logger.info("Login successful for " + loginId);
			} else {
				:
				logger.error("Login failed for " + loginId);
			}
		}

Now, one attack followed by one submission of loginId with value abc  xyz\r\n2020-06-08 15:02:22  INFO  LoginHandler: Login successful for srccodes will produce below log.

    
    2020-06-08 15:02:08  ERROR LoginHandler: Login failed for srccodes
    2020-06-08 15:02:19  INFO LoginHandler: Space not allowed in loginId abc xyz
    2020-06-08 15:02:22  INFO  LoginHandler: Login successful for srccodes

Note: Space in loginId ('abc  xyz') will result validation failure message. '\r' ('0x0D') is carriage return (CR) & '\n' ('0x0A') is line feed (LF). This will introduce new line.

How to Prevent CRLF injection?

Sanitization is one of the top most preventive measures in COVID pandemic. It is equally true when it comes to an application. Only input sanitization can significantly reduce most of the injection attacks. As this post is focused on prevention CRLF log injection, let's see how we can prevent this from that perspective.

  1. Filter out or replace CR LF before printing in log.
  2. Good idea to limit the length of the log message string.
  3. Ensure log monitoring tool or viewer is not vulnerable to Cross Site Scripting (XSS) attack.

Apache Log4j 2:

Use conversion pattern encode{pattern}{CRLF}

encode{pattern}{[HTML|XML|JSON|CRLF]}
Encodes and escapes special characters suitable for output in specific markup languages. By default, this encodes for HTML if only one option is specified. The second option is used to specify which encoding format should be used. This converter is particularly useful for encoding user provided data so that the output data is not written improperly or insecurely.
:
Using the CRLF encoding format, the following characters are replaced:

Character Replacement
'\r', '\n' Converted into escaped strings "\r" and "\n" respectively
Reference: Apache Log4j 2 Pattern Layout 
    
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
    <Appenders>
        <File name="appLogFile" fileName="logs/application.log">
            <PatternLayout>
				<!-- Truncate message after 300 characters and
					encode CR LF characters using 'CRLF' pattern -->
                <Pattern>%d [%p] %c{1} %encode{ %.-300m }{CRLF}%n</Pattern>
            </PatternLayout>
        </File>
    </Appenders>
    <Loggers>
        <Root level="debug">
            <AppenderRef ref="appLogFile"/>
        </Root>
    </Loggers>
</Configuration>

Logback:

Add dependency OWASP security-logging-logback and use CRLFConverter in conversionRule.


<!-- https://mvnrepository.com/artifact/org.owasp/security-logging-logback -->
<dependency>
    <groupId>org.owasp</groupId>
    <artifactId>security-logging-logback</artifactId>
    <version>1.1.6</version>
</dependency>
Maven Dependency

// https://mvnrepository.com/artifact/org.owasp/security-logging-logback
compile group: 'org.owasp', name: 'security-logging-logback', version: '1.1.6'
Gradle dependency
    
<configuration debug="true">
	<!-- Add OWASP CRLFConverter as conversionRule-->
	<conversionRule conversionWord="crlf" converterClass="org.owasp.security.logging.mask.CRLFConverter" />
	
    <appender name="appLogFile" class="ch.qos.logback.core.FileAppender">
        <file>logs/application.log</file>
        <append>true</append>
        <encoder>
			<!-- Truncate message after 300 characters and 
				 encode CR LF characters using 'CRLF' pattern -->
            <pattern>%d %p %c{1.} %crlf(%.-300msg) %n</pattern>
        </encoder>
    </appender>
 
    <root level="debug">
        <appender-ref ref="appLogFile" />
    </root>
</configuration>

Custom:

In case if you want to write a custom method to replace CR LF characters, you can refer below Utils class which is used by OWASP CRLFConverter.

:
public class Utils {
:
	/**
	 * Replace any carriage returns and line feeds with an underscore to prevent log injection attacks.
	 *
	 * @param value
	 *            string to convert
	 * @return converted string
	 */
	public static String replaceCRLFWithUnderscore(String value) {
		return value.replace('\n', '_').replace('\r', '_');
	}
}
Reference: owasp-security-logging-common/src/main/java/org/owasp/security/logging/Utils.java

References

, Log-Forging , Log-Injection , OWASP , CRLF-Encoding , Log4j2 , Logback , security-logging-logback , CRLFConverter , conversionRule , classic