The security considerations for file upload

The overall goal is to build a set of defensive layers that tightly control

the process of uploading the file and later retrieval of the file. The user will

always interact indirectly with the file and never directly access the file

system without application control.

This document is constructed into 2 parts, A section mentioned about

unrestricted file upload vulnerability, the one B is describe some of methods

to implement for details. Read it carefully to use them most effectively.

A- Unrestricted File Upload Vulnerability. 1. Description

Uploaded files represent a significant risk to applications. The first step in many

attacks is to get some code to the system to be attacked. Then the attack only needs to

find a way to get the code executed. Using a file upload helps the attacker accomplish

the first step.

The consequences of unrestricted file upload can vary, including complete system

takeover, an overloaded file system or database, forwarding attacks to back-end

systems, and simple defacement. It depends on what the application does with the

uploaded file and especially where it is stored.

There are really two classes of problems here. The first is with the file metadata,

like the path and file name. These are generally provided by the transport, such as

HTTP multi-part encoding. This data may trick the application into overwriting a

critical file or storing the file in a bad location. You must validate the metadata

extremely carefully before using it.

The other class of problem is with the file size or content. The range of problems

here depends entirely on what the file is used for. See the examples below for some

ideas about how files might be misused. To protect against this type of attack, you

should analyze everything your application does with files and think carefully about

what processing and interpreters are involved.

2. Risk Factors

- The impact of this vulnerability is high, supposed code can be executed in the

server context or on the client. The likelihood of a detection for the attacker is

high. The prevalence is common. As a result the severity of this type of

vulnerability is high.

- The web server can be compromised by uploading and executing a web-shell

which can run commands, browse system files, browse local resources, attack

other servers, and exploit the local vulnerabilities, and so forth. This may also

result in a defacement.

- An attacker might be able to put a phishing page into the website.

- An attacker might be able to put stored XSS into the website.

- This vulnerability can make the website vulnerable to some other types of attacks

such as XSS.

- Picture uploads may trigger vulnerabilities in broken picture libraries on a client

(libtiff, IE had problems in the past) if the picture is published 1:1.

- Script code or other code may be embedded in the uploaded file, which gets

executed if the picture is published 1:1.

- Local vulnerabilities of real-time monitoring tools, such as an antivirus, can be

exploited.

- A malicious file (Unix shell script, windows virus, reverse shell) can be uploaded

on the server in order to execute code by an administrator or webmaster later -- on

the server or on a client of the admin or webmaster.

- The web server might be used as a server in order to host of malware, illegal

software, porn, and other objects.

3. Examples

Attacks on application platform

- Upload .jsp file into web tree - jsp code executed as web user

- Upload .gif to be resized - image library flaw exploited

- Upload huge files - file space denial of service

- Upload file using malicious path or name - overwrite critical file

- Upload file containing personal data - other users access it

- Upload file containing "tags" - tags get executed as part of being "included" in a

web page

Attacks on other systems

- Upload .exe file into web tree - victims download trojaned executable

- Upload virus infected file - victims' machines infected

- Upload .html file containing script - victim experiences Cross-site Scripting

(XSS)

4. Weak Protection Methods and Methods of Bypassing

Using Black-List for Files Extensions

Some web applications still use only a black-list of extensions to prevent from

uploading a malicious file.

- It is possible to bypass this protection by using some extensions which are

executable on the server but are not mentioned in the list. (Example: file.php5,

file.shtml, file.asa, or file.cer)

- Sometimes it is possible to bypass this protection by changing some letters of

extension to the capital form (example: file.aSp or file.PHp3).

- Using trailing spaces and/or dots at the end of the filename can sometimes cause

bypassing the protection. These spaces and/or dots at the end of the filename will

be removed when the file wants to be saved on the hard disk automatically. The

filename can be sent to the server by using a local proxy or using a simple script

(example: file.asp ... ... . . .. .., file.asp , or file.asp.).

- A web-server may use the first extension after the first dot (.) in the file name or

use a specific priority algorithm to detect the file extension. Therefore, protection

can be bypassed by uploading a file with two extensions after the dot character.

The first one is forbidden, and the second one is permitted (example:

file.php.jpg).

- In case of using IIS6 (or prior versions), it might be possible to bypass this

protection by adding a semi-colon after the forbidden extension and before the

permitted extension (example: file.asp;.jpg).

- In case of using IIS6 (or prior versions), it might be possible to bypass this

protection by putting an executive file such as ASP with another extension in a

folder which ends with an executive extension such as .asp (example:

folder.asp\file.txt). Besides, it is possible to create a directory just by using a

file uploader and ADS (Alternate Data Stream). In this method, filename should

end with ::$Index_Allocation or :$I30:$Index_Allocation to create a directory

instead of a file (example: newfolder.asp::$Index_Allocation creates

newfolder.asp as a new directory).

- This protection can be completely bypassed by using the e.g. control characters

like Null (0x00) after the forbidden extension and before the permitted one. In this

method, during the saving process all the strings after the Null character will be

discarded. Putting a Null character in the filename can be simply done by using a

local proxy or by using a script (example: file.asp%00.jpg). Besides, it would

be perfect if the Null character is inserted directly by using the Hex view option

of a local proxy such as Burpsuite or Webscarab in the right place (without

using %).

- It is also possible to create a file with a forbidden extension by using NTFS

alternate data stream (ADS). In this case, a : sign will be inserted after the

forbidden extension and before the permitted one. As a result, an empty file with

the forbidden extension will be created on the server (example: file.asp:.jpg).

Attacker can try to edit this file later to execute his/her malicious codes. However,

an empty file is not always good for an attacker. So, there is an invented method

by the author of this paper in which an attacker can upload a non-empty shell file

by using the ADS. In this method, a forbidden file can be uploaded by using this

pattern: file.asp::$data..

- In Windows Servers, it is possible to replace the files by using their short-name

(8.3). (example: web.config can be replaced by uploading web~1.con)

- Sometimes combination of the above can lead to bypassing the protections.

Using White-List for Files Extensions

Many web applications use a white-list to accept the files extensions. Although

using white-list is one of the recommendations, it is not enough on its own. Without

having input validation, there is still a chance for an attacker to bypass the protections.

- the 3rd, 4th, 5th, and 6th methods of last section apply here as well.

- The list of permitted extensions should be reviewed as it can contain malicious

extension as well. For instance, in case of having .shtml in the list, the

application can be vulnerable to SSI attacks.

Using Content-Type from the Header

Content-Type entity in the header of the request indicates the Internet media

type of the message content . Sometimes web applications use this parameter in order

to recognize a file as a good one. For instance, they only accept the files with the

Content-Type of text/plain.

- It is possible to bypass this protection by changing this parameter in the request

header by using a local proxy.

Using a File Type Recognizer

Sometimes web applications intentionally or unintentionally use some functions (or

APIs) to check the type of the file in order to do further process. For instance, in case

of having image resizing, it is probable to have image type recognizer.

- Sometimes the recognizers just read the few first characters (or header) of the

files in order to check them. In this case, an attacker can insert the malicious code

after some valid header.

- There are always some places in the structure of the files which are for the

comments section and have no effect on the main file. And, an attacker can insert

malicious codes in these points.

- Also, it is not impossible to think about a file modifier (for example an image

resizer) which produces malicious codes itself in case of receiving special input.

5. Prevention Methods (Solutions to be more secure)

Some special recommendations for the developers and webmasters:

- Never accept a filename and its extension directly without having a white-list

filter.

- It is necessary to have a list of only permitted extensions on the web application.

And, file extension can be selected from the list. For instance, it can be a select

case syntax (in case of having VBScript) to choose the file extension in regard to

the real file extension.

- All the control characters and Unicode ones should be removed from the

filenames and their extensions without any exception. Also, the special characters

such as ;, :, >,

name and also the extension should not be empty at all (regular expression: [a-zA-

Z0-9]{1,200}\.[a-zA-Z0-9]{1,10}).

- Limit the filename length. For instance, the maximum length of the name of a file

plus its extension should be less than 255 characters (without any directory) in an

NTFS partition.

- It is recommended to use an algorithm to determine the filenames. For instance, a

filename can be a MD5 hash of the name of file plus the date of the day.

- Uploaded directory should not have any execute permission.

- Limit the file size to a maximum value in order to prevent denial of service

attacks (on file space or other web applications functions such as the image

resizer).

- Restrict small size files as they can lead to denial of service attacks. So, the

minimum size of files should be considered.

- Use Cross Site Request Forgery protection methods.

- Prevent from overwriting a file in case of having the same hash for both.

- Use a virus scanner on the server (if it is applicable). Or, if the contents of files

are not confidential, a free virus scanner website can be used. In this case, file

should be stored with a random name and without any extension on the server

first, and after the virus checking (uploading to a free virus scanner website and

getting back the result), it can be renamed to its specific name and extension.

- Try to use POST method instead of PUT (or GET!)

- Log users activities. However, the logging mechanism should be secured against

log forgery and code injection itself.

- In case of having compressed file extract functions, contents of the compressed

file should be checked one by one as a new file.

B- Guideline for implement: Upload all files to a file-share and not directly to database as a blob:

If the files are uploaded to the database as a blob then the data that resides

on the file will not be validated thus may lead to injection attacks. For example an

attacker may use this vulnerability to inject malicious data in to the database and

use it later to perform other attacks. So, it is always a better way to upload the

files to a file-share instead of the database as a blob.

In general, try to store client-supplied files on their own partition. Avoid storing them on the same partition as the web site, and always avoid storing

them on the system drive:

- An attacker might be able to exhaust free space on the system drive or the web

site drive, resulting in unexpected behavior by the OS or the web site.

Alternately, if the attacker can do a directory-traversal attack, then the attacker

might be able to replace system or web site files, thereby compromising the

web server system.

- Allowing a user to upload the files either to a system drive or website partition

may lead to DOS attacks.

If client-supplied files are stored on the web site partition, make sure that they never reside within the web site directory tree:

Allowing client-supplied files in the web site directory tree might result in

an attacker being able to upload a malicious file, and then navigate to it using an

HTTP GET request. This would cause the server to execute the attacker-supplied

page using the identity of the web application account.

Alternately, an attacker might upload a file containing malicious script, and then

lure a victim user to access the same file via GET, effectively resulting in a cross-

site script exploit.

The HTTP POST method is preferable to the HTTP PUT method: When using the PUT method, you need to take extra care to ensure that

user-supplied files cannot be accessed with other methods such as GET. This

would result in the same cross-site script attack described above.

Scan client-supplied files for viruses using a well - known industry standard virus scanner:

Allowing client-supplied files to be uploaded to a server without a virus

scan might result in an attacker being able to upload malicious files such as

viruses.

Make sure that the well - known industry standard virus scanner is used on

the server and the virus signatures are up to date.

Ensure all incoming/outgoing Files are scanned and they are scanned on real time

basis.

Never assume that files which pass a virus scan are completely safe, they

may contain a malicious code that could help the attacker to perform other attacks

like code injection.

Check and restrict the size of uploaded files if possible, and allocate server-side resources sparingly.

- Attackers who can upload extremely large files can potentially deplete or

exhaust server resources, resulting in unexpected behavior by the server or

web site.

- Ensure application checks for the length so that it doesnt potentially deplete

or exhaust server resources.

When storing client-supplied files on disk, generate your own file paths and file names. Do not allow user-supplied data to influence the path or name of

the file as it is actually stored on disk:

- By depriving attackers of the ability to influence how your application

performs file-level I/O, you all but eliminate a number of possible attacks,

such as directory traversal, induced filename collisions, and more.

- If it is necessary to store the file on disk using a user-supplied name, you

should do the following:

a. Use the Path.GetFilename() method to safely retrieve a valid filename from

the client-supplied value.

b. Use a directory path you choose yourself, without allowing user input to

influence the directory path.

Lock down access to specific users and security-groups requiring access to the upload share.

Treat client-supplied files like any other element of user input and validate, validate, validate.

- Verify that the file is of the type the user is permitted to upload.

- Discard any file which does not have an extension or name that is explicitly

allowed or expected by your application.

- Also consider inspecting the internal contents of the file to ensure that it is of

the type advertised by the user-supplied name or extension.

Finally: Remember the #1 rule of good web application security, see it below. All users are

evil!

The following basis rules to Secure File Uploads, Control File Upload are introduced

below, based summarization all of principle to secure file upload by users:

1. Create a new file name

Do not use the user supplied file name as a file name on your local system. Instead,

create your own unpredictable file name. Something like a hash (md5/sha1) works as it is easily validated (it is just a hex number). Maybe add a serial number or a time stamp to

avoid accidental collisions. You may add a secret to the name to make it harder to guess the file name. If you need to keep the original file name: use a look-up table to link the validated user supplied file name to the server created name.

2. Store the file outside of your document root If your document root is /var/www/html, create a directory /var/www/uploads and use it

to store uploaded files. That way, an attacker will not be able to retrieve the file directly. This will allow you to provide fine grained access control. The file will not be parsed by the server's application language module but the source of the file will be streamed. Also,

we should check the directory property is not execute , forbid any execute files from users.

3. Check the file size

You should set a maximum file size in the upload form, but remember: It is just advisory. Make sure to check the file size after the upload completed. Be in particular careful if you

allow the upload of compressed files and later uncompress them on the server. This scenario is very hard to secure.

4. Extensions are meaningless However, even Apache doesn't always behave the way you expect it to. Try

'something.php.x' in Apache and chances are that php code will be executed. Its a feature. If you stream a file back to the user, the extension isn't what matters, but the Content-

Type header and the file's header. It is best to use the "file" command on unix to check the file type. But even this is not fool proof. It will just check the first few bytes. In PHP for example, a file may start with a GIF header, but later if the PHP engine sees a

"

6. Keep tight control of permissions Any uploaded file will be owned by the web server. But it only needs read/write

permission, not execute permissions. After the file is downloaded, you could apply additional restrictions if this is appropriate. Sometimes it can be helpful to remove the

execute permission from directories to prevent the server from enumerating files.

7. Authenticate file uploads File uploads, in particular if these files are viewable by others without moderator review,

have to be authenticated. This way it is at least possible to track who uploaded an objectionable file.

8. Check magic number of file type. Reference: http://www.garykessler.net/library/file_sigs.html

http://en.wikipedia.org/wiki/Magic_number_(programming)

9. Limit the number of uploaded files

Many developers limit the file size, but not all limit the number of files uploaded in a

request. Make sure to apply reasonable limits. But be also ready for a DoS attack that just

uploads a large number of small files. Pick an appropriate directory structure to limit the

number of files per directory and pick an appropriate file system.