Jump to content
Larry Ullman's Book Forums

Uploading Images With Type Set As Application/Octet-Stream


Recommended Posts

This might be of interest to many in this forum who want to do image uploads. I am working on a site where users can upload a profile image and I am essentially using Larry's image upload code as written. The site is responsive and has to allow image uploads from smart phones as well. The problem is that when I tried uploading images from my Android phone, I kept getting a "The uploaded file was not of the proper type." error. I then put in an echo statement to display the file type and MIME type and found out that the images had a type of "application/octet-stream" and a MIME of "image/jpeg", and since we are checking both the type and MIME type, that is why the upload failed.

 

This is a major problem for smart phone users, as many Android devices set the type of images taken with the onboard camera to "application/octet-type". I know that application/octet-stream is a generic type and could represent any binary data, including a malicious file. However, the web is a constantly changing place, and we need to be flexible in order not to marginalize users. This is especially true in the case of smart phones, as they now represent a greater share of devices used to access the Internet than PCs. Telling a user that they can't upload an image because they have the wrong device is unacceptable!

 

Is there a good solution that will allow greater flexibility in the types of images that can be uploaded, while still maintaining security?

 

I found a solution on S.O. that might be acceptable:

//if mime type is application/octet-stream (psp gives jpegs that type) try to find a   more specific mime type
$mimetype = strtolower(preg_replace("/^(.+?);.*$/", "\\1", $_FILES['form_field']  ['type'])); //reg exp copied from CIs Upload.php
if($mimetype == 'application/octet-stream'){
   $finfo = finfo_open(FILEINFO_MIME, '/usr/share/file/magic');
   if($finfo){
   $_FILES['form_field']['type'] = finfo_file($finfo, $_FILES['form_field']['tmp_name']);
   finfo_close($finfo);
   }
   else echo "finfo_open() returned false");
} 

 

It basically gets the MIME type and sets the file type to be the same.

The problem is that the Fileinfo extension needs to be installed on the server.

 

If anyone has any thoughts on this, it would be greatly appreciated!

 

Thanks in advance,

 

Matt

  • Upvote 1
Link to comment
Share on other sites

Interesting. I was not aware of this myself. First of all, what's wrong with using the Fileinfo approach? Does your server not support that? Second, you can improve the security by how you present the uploaded file back to the users, just as you would with any uploaded file. By that I mean using the IMG tag, using a proxy script, setting the right headers, etc.

Link to comment
Share on other sites

Larry,

 

Thank you so much for responding to my post! I know you take security very seriously, and thanks to reading your books, so do I! I think HartleySan probably gets annoyed with my stubborn approach to it sometimes, but nothing can be left to chance.

 

I heard that as of PHP 5.3.0 the Fileinfo extension is installed by default (prior to PHP 5.3.0, it was a PECL extension which had to be installed manually) and my server is using PHP 5.3.13, so it should be fine.

 

Also, I looked more into the image upload security issue and found the following website. It is fairly recent and shows the seriousness of the situation.

 

http://nullcandy.com/php-image-upload-security-how-not-to-do-it/

 

The most interesting part is the one with the cute little image of the kitten... with PHP code embedded! It wouldn't matter if you accessed it with a proxy script and it was stored outside the web root, or on a server on another planet (although that might not be a bad idea), when the image is displayed, that code WILL execute! Download the image and look at it in a text editor. Larry, my heart sank into my stomach when I saw that!

 

There are a couple of solutions to overcome the preceding problem:

 

1) Re sample all the images using an image library when you move them from the temp folder to the destination folder. The downside to this is that it won't take long for a hacker to figure out what you are doing and the GD library can be exploited by code in the image.

 

2) Put an htaccess file in the folder which prevents executables from running.

 

As you mention in the book, the initial checks for file extension, type, and MIME can all be worked around as well. As one S.O. poster said "Hackers are bored by this!" However, they are still very important and should be strictly checked. HartleySan also suggested running the original file name through a regex, and that is a solution which is strongly suggested by many sites. I found a good whitepaper about file upload security: https://www.owasp.org/index.php/Unrestricted_File_Upload A lot of the points are already covered in your book, but there are a few others also.

 

Larry, if you or anyone else has any more input or ideas, please post them, as this issue is very serious and will help others in this forum.

 

Matt

  • Upvote 1
Link to comment
Share on other sites

That kitten example is completely misleading. Otherwise it's a good article, but that example is completely misleading. Look at the source code of the page. Yes, there may be PHP code within that image, but that doesn't mean it's dangerous. For that kitten image to be displayed on the page, a regular IMG tag is used and the file has a .jpeg extension. Absolutely nothing dangerous there, whether or not a proxy script is in use. The link surrounding the kitten image is to kitten.jpeg.php, which executes the code written into the image (in theory, hard to say considering the fakery already going on in the article). So even if PHP code is written into an image, that's only a problem if the image:

 

A) Has a .php extension

B ) Is executed as PHP

 

These are very easy to prevent. Scary, yes, but entirely misleading.

Link to comment
Share on other sites

@Jonathan - Yes, I am absolutely using a proxy script!

 

For that kitten image to be displayed on the page, a regular IMG tag is used and the file has a .jpeg extension. Absolutely nothing dangerous there, whether or not a proxy script is in use. The link surrounding the kitten image is to kitten.jpeg.php, which executes the code written into the image (in theory, hard to say considering the fakery already going on in the article).

 

Thank you Larry! That is very reassuring to know that it's not as dangerous as it would seem. Yeah, the article was good, but you're right that it is a little misleading in some places. I had to read a couple sections twice to understand exactly what he was talking about. He could have given more details on a few of the points to clarify what exactly was going on.

 

I just had couple more questions about image uploads, and these mainly have to do with folder security and the proxy script.

 

1 ) I have read in various places that it is good to put an .htaccess file into the uploads folder which prevents executables from running. The problem is that I have come across 3 different types.

 

A ) This one comes from the article with the kitten:
ForceType application/octet-stream
<FilesMatch "(?i)\.jpe?g$">
ForceType image/jpeg
</FilesMatch>
<FilesMatch "(?i)\.gif$">
ForceType image/gif
</FilesMatch>
<FilesMatch "(?i)\.png$">
ForceType image/png
</FilesMatch>

 

B ) This is simply a "blacklist" of files not to be executed from within the folder:
_flag engine 0
RemoveHandler .phtml .php .php3 .php4 .php5 .php6 .phps .cgi .exe .pl .asp .aspx .shtml .shtm .fcgi .fpl .jsp .htm .html .wml
AddType application/x-httpd-php-source .phtml .php .php3 .php4 .php5 .php6 .phps .cgi .exe .pl .asp .aspx .shtml .shtm .fcgi .fpl .jsp .htm .html .wml

 

C ) Prevent anything that's not am image from executing within the folder:
Order Deny,Allow
Deny from all[/indent]

[indent=1]<FilesMatch "\.(gif|jpe?g|png)$">
Order Deny,Allow
Allow from all
</FilesMatch>

 

Which one do you think is the best for stopping scripts from being run in the folder?

 

2 ) I know in PHP6 And MYSQL5 you pass the picture id to the proxy script to get the image. I am using the new image name instead (i.e. jfv399u325ut847y34y.jpg). I then run it through a regex in the proxy script to make sure that it contains only alpha-numeric characters and has either a .jpg, .png, or a .gif extension. I don't think this matters as the image is stored outside the web root and the user can access it anyway. What do you think?

 

3 ) I also want to use a mod re_write to hide the proxy script from the user, so this...

<img src="http://www.example.com/users/show_image.php?image=546dd6992d97e625461ccb53fe062363526db65e.jpg&name="546dd6992d97e625461ccb53fe062363526db65e.jpg" alt="Matt" >

 

becomes...

<img src="http://www.example.com/users/img/546dd6992d97e625461ccb53fe062363526db65e.jpg" alt="Matt" >

 

Is this a good idea?

 

Sorry for the long post, but I just want to make sure I am following a good preventative approach to stop any potential security breaches!

 

Thanks again for your help!

 

Matt

  • Upvote 1
Link to comment
Share on other sites

Larry,

 

Thank you so much for your advice!  I will go ahead and use these in my site design.

 

I just had one last question regarding getting the correct image type.  As we know, using $_FILES['type'] to determine a file's correct image type is almost completely useless as it is determined by the browser, and thus, completely able to be overwritten by the user.  As such, I wanted to know if it was ok to use getimagesize() to do the same thing?

 

I was thinking of doing something like this:

$image_info = @getimagesize($file['tmp_name']);

$image_type = $image_info[2];

$valid_types = array(IMAGETYPE_GIF, IMAGETYPE_JPEG, IMAGETYPE_PNG);

if(!(in_array($size[2], $valid_types))) {

  $add_image_errors['image'] = 'The uploaded file was not of the proper type.';

}

 

What do you think?

 

Thanks again!

 

Matt

Link to comment
Share on other sites

 Share

×
×
  • Create New...