Matt

Laurent, maybe I can help with this as I've recently created a massive update form based on a registration form that HartleySan wrote for the same site. First of all, when the user accesses the page, the form needs the previous values already stored in the database. The way HartleySan and I go about this is like this: if ($_SERVER['REQUEST_METHOD'] === 'POST') { Assign $_POST values to variables and validate them. ex.) $email = $_POST['email']; if (is_empty($errors)) { update the database } else { Get the original values from the database and assign them to variables. ex.) mysqli_stmt_bind_results($stmt, 's', $e); $email = $e; } Create the form and pass in the variables to each input. ex.) <input id="email" name = "email" type="email" value="<?php echo $email>"> The point is that you can't do things exactly the same way that Larry does with a usual insert form. You have to change the "sticky form" logic from within the form functions and replace them with simple echos of a value that you pass to the functions as a parameter. As an example: function makeTextInput($id, $label, $default = '', $errors, $type = 'text') { $error = false; foreach ($errors as $k => $v) { if ($k === $id) { $error = true; break; } } $return_str = '<label for="' . $id . '" class="input-label-top'; if ($error) { $return_str .= ' error_input_label'; } $return_str .= '">' . $label . '</label> <input type="' . $type . '" id="' . $id . '" name="' . $id . '"'; $return_str .= ' value="' . htmlspecialchars($default) . '"'; $return_str .= ' class="text_input'; if ($error) { $return_str .= ' error_input'; } $return_str .= '">'; if (isset($errors['form_errors'])) { if (array_key_exists($id, $errors)) { $return_str .= '<p class="form_error_message">' . $errors[$id] . '</p>'; } } return $return_str; } echo makeTextInput('email', 'Email Address', $email, $errors); I don't know if this is what you are looking for, but hopefully it gets you on the right path! Matt

Larry, Thanks for your response! What do you mean that output buffering is built into php by default? Do you mean that in php 5.5 we don't have to use ob_start() to prevent headers already sent messages? I agree with you about writing better code, and I'm not trying to make you angry, but we were erm... using your code. I wrote a comment because I got the same error as 100yen. However, in many cases I think that preventing the headers already sent error leads to "worse code" because most of the time you have to re-factor everything with a half-ass solution to get around it! I'm sorry, but doing something as simple as regenerating a session ID isn't what I would consider writing bad code, and in any rational server-side language, something as ridiculous as this would not be a problem. I love php, but it really has some "dirty" areas that need to be cleaned up!

Larry, I also get tired of dealing with the "headers already sent" stuff! Is there any problem using ob_start() at the beginning of every page to just get around all this? If there isn't, then why isn't it just built into php by default? Thanks, Matt

Laurent, Glad you were able to figure it out! I have been through the same frustrating situations as you many times and it always comes down to some careless oversight (and it's usually the last place you think the problem is too). Setting the correct column type and length for a field storing a hashed password is very important if you want to get back the same value that you originally put in the database. Enjoy the rest of the book! Matt

Laurent, This is Larry's code verbatim, and I know it works, so the problem could be with how the password is originally stored. Do you have the pass field set up as a varchar (255)? Are you using Larry's code (i.e. the BCRYPT algorithm) to store the password? Also, what version of php are you running? Matt

Antonio, Thank you very much for the informative advice! I agree with you that obscuring the id doesn't do anything to improve security and will use more processing to encode/decode it. In our case, only paid customers can "add" users, so users can't add other users. Also, there limit to the number of users a customer can add, so that won't cause any problems. Thanks again, Matt

Larry and Antonio, Thanks for the nice comments, but I think Jon deserves almost all the credit for it right now. He built the site (quite quickly I must add) and wrote the first post, but I did provide feedback and support. My biggest contribution was probably acting as the initial inspiration for it as I was tired of going to S.O. and other sites looking for answers to certain web related questions, only to read a myriad of half informed, or downright wrong responses to the topic. You know the kind of people who, when the OP asks something about mysqli, some arrogant a-hole tells him he should be using PDO because, well, it's easier and more powerful. Of course, nowhere did he actually answer the question (and he also doesn't seem to have any understanding of the drawbacks of using PDO, just that it's OOP and everyone else is using it, so it must be the "thing" to use)! I know Jon was getting tired of this as well. We wanted to create a one-stop site where all off the best solutions for problems we have discovered over the years (and this includes a lot of Larry's as well) could be documented in one place for anyone to download and use. Anyone who wants to contribute, and has discovered a good solution for a specific problem, is more than welcome to submit a post to it! Matt

Larry, I just had a quick question. I remember you saying that it is not recommended to show a user's ID on a page, but I was wondering why this is so bad? If you have taken the necessary precautions like validation, using prepared statements, aren't concatenating strings in queries, etc... then there is absolutely no way that a user could do anything to manipulate his record in the database. Second, I think that in certain situations displaying the user's ID (or at least a unique ID that can be connected to that user) is almost necessary. For the site Jon and I are working on now, customers can add users to a private list. When the customer is adding a user, they must click on a link next to the user's picture. This could not be achieved without embedding some kind of unique ID in the link that is associated with the user. The only other solution I can think of to try and hide the user ID would be to add integers of a predefined length to both ends of the user's actual ID and then strip them off when it has to be processed. Example: user id: 00237 Number displayed in the link: 6253623774838 When the id needs to be retrieved, the 4 numbers at the beginning and end are stripped off and the remaining number is padded with zeros. Another way would be to encode the date the user registered and use that instead. However, all of this seems a bit convoluted to me! What do you think? Thanks, Matt

Jon, Thanks for the hard work you have done with this! The functions look great and include most of the best security practices for dealing with sessions. I only had one question regarding the session_init() function. Why did you leave out a property for specifying whether the connection is https? You can then use that parameter to set whether the cookie is being sent over https or not like in the Treehouse Blog tutorial. I have read that it is better to explicitly set all the cookie arguments when the session is created and to set HttpOnly to true as it's more secure. What are your thoughts on this?

This has been a very interesting read! I want to throw my 2 cents in as well, and either of you can correct me if I'm wrong. Even though I have had a lot of experience coding, both of you guys are better coders than me! I love database design and programming as well as the front end, so I tend to spread myself too thin I think. That being said, I have learned a lot from Jon over the past 2 years, and let me tell you, when it comes down to it we are a lot alike in our attitudes towards coding. For one, I agree with Jon that the iterative style of testing is better than throwing a bunch of code out on the screen, and then going back to fix all the bugs. Many programmers (including myself) often don't take the time to go through and thoroughly think through the logic they have written. Second, over time I have become much more flexible about my approach towards solving problems in the most organized way. What I mean is that I will often do things a little outside accepted patterns in order to achieve better performance. One example being that I know very well how to normalize a database, but when you start thinking about how the data will be accessed, sometimes some redundant data in a table is better than 6 joins, which will slow down the site and irritate the user! The same is true for procedural vs. OOP! While OOP is an awesome way to encapsulate and organize code, it has it's time and place and procedural programming will always be faster (and easier to debug IMHO). I know that Jon agrees with me on this! The same goes with frameworks. Why in the hell would I trust my site's security to a huge entanglement of classes and bloat when I can code the same thing in a little more time, but have it twice as secure, take a third the space on the hard drive, and be 10 times as fast? I'm not saying that I don't use someone else's class or library here and there, but the point is that I am not going to rely on another person's code to make my site. Sorry guys, I have vented enough! Also, Antonio, I know as well that you are an awesome coder and you have given me some good advice here in the past. I'd like to hear your take on the whole hardcore MVC, let's recode our whole site using classes up the ying yang, trend? Also, having looked at your excellent work over the years, you would be a great choice to give Jon and I some advice on ways to better organize our site. Matt

@Antonio - Thanks for sharing that session component. I agree that using something that already exists, and not completely reinventing the wheel, is the best way to go. I will look at it! I have looked into how some other frameworks handle session management and wasn't too impressed. For example, with the Zend session component, you have to use 3 or 4 classes to do what could easily be done with one. @Jon - Thanks for taking on this endeavor. One thing I wanted to say was that in my research I found that there are a few small things to watch out for when developing your own session class, and I do think it is a good idea to use others as a model when creating one. For example, whenever you regenerate a session ID, it can cause race conditions in sites that use a lot of AJAX. There is an excellent tutorial from the Treehouse blog here: http://blog.teamtreehouse.com/how-to-create-bulletproof-sessions After stripping out the code which tries to lock the session to the same IP and User Agent, it is a great base class to build on. It also has an nice function for regenerating session IDs which takes care of the problem I mentioned above. One guy said that you should use a static class for the session manager because only one should ever be created per user per visit. Does this have any benefit over using a regular class to handle sessions? Also, I have read that if you have a small site which frequently accesses the database (as ours does), then storing session data in it can cause performance problems, as now the database will need to be accessed every time a session is created, written to, or read from. It is probably fine in most cases, but just something to think about.

Larry, Thanks for responding to this thread! I have been looking into session security for a couple weeks and have talked to HartleySan about it. I feel that we absolutely should be working to secure sessions as much as possible as these types of attacks are increasing. I read the same thing! Also, because service providers like AOL use shared IP pools, and an IP address can change mid connection, checking for IP addresses is problematic, unless you write a work around for those using AOL, or anyone else using a similar service. As for checking the user agent, I've heard that this is completely useless, and serves nothing more than a way to give the developer a false sense of security! I think that a session class is an excellent candidate for using objects as it will encapsulate all the session handling in a convenient place. We could all write one here in the forum. Anyone who wants to contribute would be welcome and we all could use it! Any thoughts? Matt

Jon, I agree that using a transaction is the best way to do it! Doing all the queries one after the other, and rolling back if any of them fails, makes sense. Also, for non-JS users, doing the whole form in one submission is great. Adding Ajax requests only to the train data is a whole different story, as we would also have to deal with the rest of the form submission separately. What do you think?

I wasn't here for a couple of days and all hell breaks loose! Jon, yes you understood my question correctly! I think I have a solution for the updates. I think you're right about having them in series, but the problem is that if any one query fails, then only certain sections will get updated. This may confuse the user, so I decided it is better to keep track of errors at each step. This way we can rollback if anything fails. Any thoughts? Antonio, please don't worry at all about any misunderstandings. I know you are a great coder and value your input highly! As far as the form, Jon and I discussed the implementation of the train stations in depth before he coded it and it's probably about as intuitive as you can make it! I had spent several days studying web form design best practices, and everything from the line spacing to the size of the buttons has been taken into account as well. Matt

Thank you for the advice guys! I really appreciate it! The next problem I have to deal with is all the nested table updates. Only one uses an actual UPDATE query, but the rest use DELETE/INSERT to update the data (because they are many-to-many relations). Because the user probably won't update data that will affect every table, I can't use mysqli_stmt_affected_rows() to check whether a specific update was successful or not (and move on to the next query). Any thoughts on this? Thanks, Matt

Thank you very much for the responses! @Antonio - I really like the idea of using AJAX to update the train data. If a user is going to add/delete train stations where they can work, in most cases it will only be a relatively small amount as they would have already chosen most of them when they filled out the registration form. The AJAX approach sounds clean and fast (if a check box is unchecked then delete it, and if it is checked then insert it). My only concern with this is that it would still require a non-javascript solution for those who have js disabled. And you are right that we are not dealing with a large amount of stations, although a user could theoretically add hundreds if they wanted to. The site is in Japan, and as you may know, trains are the most common mode of transportation here. In a metropolitan area like Tokyo, there are probably over a hundred stations from a person's home station within an hour's commute, but I think in most cases a user won't select that many. btw - Hartley-san wrote all the backend logic for the original registration form! He did a brilliant job on it! I am taking the same form and turning it into an update form. If you want to see the original form I can send you a PM with the address and directory username/password. @Larry - I am starting to lean towards the array approach as well! I think it is probably better because there could conceivably be lots of records in the database for a given user, and deleting all of them and reinserting them just to add one train station is absurd. Either way, the whole index is going to have to be rebuilt every time the table is updated, and this will certainly slow down the query a bit. Is this something I should be worried about? Thanks again, Matt

Larry, Happy New Year! I know my posts are often not directly related to the topics in a particular book, but this one is definitely an "advanced" topic, and one which needs to be dealt with by a credible individual (there are scattered S.O. posts on the subject, but none of which provides a definitive solution for the problem). This is a conundrum that has been bothering me for a while, but I want a best practice for updating a many-to-many table involving check box data. Basically, Jon and I are working on an update form where users can add/delete train stations where they want to work (the stations appear in a list and users can check/un-check the stations they want to work at). In the user_stations table we store the user_id, the train_line_id, and the station_id. I have done a lot of research on this and there are a couple of different ways of dealing with updating a many-to-many check box table. 1) The easiest way is simply to delete all rows in the user_stations table where the user_id is equal to the logged in user's id and then insert the new train station data for the given user. This way seems to be the cleanest and most secure, but it would mean erasing everything and rewriting it again which could add unnecessary workload to the database. 2) A clever solution I have seen is to take the array of data as it was originally and compare it to the data which is being updated. All data which was in the original array, which is not in the new array, is then added to a "delete" array. All data which is in the new array, which was not in the original array, is added to the "add" array. Then, two queries are executed: one DELETES all data which is in the "delete" array, and another which INSERTS all data which is in the "add" array. This way seems more efficient, but requires a lot more processing on the PHP side. What is the best way to update a many-to-many table with check box data? This is something that I think a lot of other people would also benefit a lot from, and I am shocked that so little information is out there! As a side note - Larry has done an awesome job of explaining almost every area of PHP/mysql web development there is, and I can think of nobody better to come up with a workable solution (i.e. a detailed and comprehensive blog post) to this problem Thanks in advance, Matt

Larry, Thank you very much for the valuable feedback! Feel free to recommend changes if you think it will improve the code! Thanks again! Matt

Larry, Sorry I haven't been on here for a while, but I have been working on a project (with Hartleysan) and I have been very busy! I saw the Northeast PHP videos and your presentations were great as usual! I had a question about complicated redirects, since this is kind of an advanced topic, I decided to post it here. I am developing a site where there are customers that register, and they have clients who also register on a separate area of the site. Being very stringent about security, I like to control exactly where different users are allowed to go on the site. If a non-logged-in user tries to access a page which is meant for logged-in users, a page telling them that they have accessed it in error is simply not acceptable. I want to guide the user to where they are supposed to be. So, I have created a function which redirects invalid users in the config.php file. Here is the code: function allow_users($allowed) { switch($allowed) { case 'CLIENTS': if (!isset($_SESSION['client_id'])) { redirect_invalid_user('index.php', 'http://'); break; } case 'CLIENTS_VISITORS': if (isset($_SESSION['customers_id'])) { redirect_invalid_users('customers/index.php', 'http://'); break; } case 'CUSTOMERS': if (!isset($_SESSION['customer_id'])) { redirect_invalid_users('customers/index.php', 'http://'); break; } case 'CUSTOMERS_VISITORS': if (isset($_SESSION['client_id'])) { redirect_invalid_users('index.php', 'http://'); break; } case 'VISITOR': redirect_invalid_users('index.php', 'http://'); break; } } function redirect_invalid_user($destination = 'index.php', $protocol = 'http://') { $url = $protocol . BASE_URL . $destination; header("Location: $url"); exit(); } I would call the function on each page, passing in the argument for allowed visitors: allowed_users('CLIENTS_VISITORS'); For example, if a logged-in customer tries to access the customer registration form, he/she would be redirected back to the customer/index.php page. Is this an acceptable approach? Is there a better way of handling this? Any help or advice would be greatly appreciated! Thanks, Matt

Larry, Thank you so much for your advice! I will go ahead and use these in my site design. I just had one last question regarding getting the correct image type. As we know, using $_FILES['type'] to determine a file's correct image type is almost completely useless as it is determined by the browser, and thus, completely able to be overwritten by the user. As such, I wanted to know if it was ok to use getimagesize() to do the same thing? I was thinking of doing something like this: $image_info = @getimagesize($file['tmp_name']); $image_type = $image_info[2]; $valid_types = array(IMAGETYPE_GIF, IMAGETYPE_JPEG, IMAGETYPE_PNG); if(!(in_array($size[2], $valid_types))) { $add_image_errors['image'] = 'The uploaded file was not of the proper type.'; } What do you think? Thanks again! Matt

@Jonathan - Yes, I am absolutely using a proxy script! Thank you Larry! That is very reassuring to know that it's not as dangerous as it would seem. Yeah, the article was good, but you're right that it is a little misleading in some places. I had to read a couple sections twice to understand exactly what he was talking about. He could have given more details on a few of the points to clarify what exactly was going on. I just had couple more questions about image uploads, and these mainly have to do with folder security and the proxy script. 1 ) I have read in various places that it is good to put an .htaccess file into the uploads folder which prevents executables from running. The problem is that I have come across 3 different types. A ) This one comes from the article with the kitten: ForceType application/octet-stream <FilesMatch "(?i)\.jpe?g$"> ForceType image/jpeg </FilesMatch> <FilesMatch "(?i)\.gif$"> ForceType image/gif </FilesMatch> <FilesMatch "(?i)\.png$"> ForceType image/png </FilesMatch> B ) This is simply a "blacklist" of files not to be executed from within the folder: _flag engine 0 RemoveHandler .phtml .php .php3 .php4 .php5 .php6 .phps .cgi .exe .pl .asp .aspx .shtml .shtm .fcgi .fpl .jsp .htm .html .wml AddType application/x-httpd-php-source .phtml .php .php3 .php4 .php5 .php6 .phps .cgi .exe .pl .asp .aspx .shtml .shtm .fcgi .fpl .jsp .htm .html .wml C ) Prevent anything that's not am image from executing within the folder: Order Deny,Allow Deny from all[/indent] [indent=1]<FilesMatch "\.(gif|jpe?g|png)$"> Order Deny,Allow Allow from all </FilesMatch> Which one do you think is the best for stopping scripts from being run in the folder? 2 ) I know in PHP6 And MYSQL5 you pass the picture id to the proxy script to get the image. I am using the new image name instead (i.e. jfv399u325ut847y34y.jpg). I then run it through a regex in the proxy script to make sure that it contains only alpha-numeric characters and has either a .jpg, .png, or a .gif extension. I don't think this matters as the image is stored outside the web root and the user can access it anyway. What do you think? 3 ) I also want to use a mod re_write to hide the proxy script from the user, so this... <img src="http://www.example.com/users/show_image.php?image=546dd6992d97e625461ccb53fe062363526db65e.jpg&name="546dd6992d97e625461ccb53fe062363526db65e.jpg" alt="Matt" > becomes... <img src="http://www.example.com/users/img/546dd6992d97e625461ccb53fe062363526db65e.jpg" alt="Matt" > Is this a good idea? Sorry for the long post, but I just want to make sure I am following a good preventative approach to stop any potential security breaches! Thanks again for your help! Matt

Larry, Thank you so much for responding to my post! I know you take security very seriously, and thanks to reading your books, so do I! I think HartleySan probably gets annoyed with my stubborn approach to it sometimes, but nothing can be left to chance. I heard that as of PHP 5.3.0 the Fileinfo extension is installed by default (prior to PHP 5.3.0, it was a PECL extension which had to be installed manually) and my server is using PHP 5.3.13, so it should be fine. Also, I looked more into the image upload security issue and found the following website. It is fairly recent and shows the seriousness of the situation. http://nullcandy.com/php-image-upload-security-how-not-to-do-it/ The most interesting part is the one with the cute little image of the kitten... with PHP code embedded! It wouldn't matter if you accessed it with a proxy script and it was stored outside the web root, or on a server on another planet (although that might not be a bad idea), when the image is displayed, that code WILL execute! Download the image and look at it in a text editor. Larry, my heart sank into my stomach when I saw that! There are a couple of solutions to overcome the preceding problem: 1) Re sample all the images using an image library when you move them from the temp folder to the destination folder. The downside to this is that it won't take long for a hacker to figure out what you are doing and the GD library can be exploited by code in the image. 2) Put an htaccess file in the folder which prevents executables from running. As you mention in the book, the initial checks for file extension, type, and MIME can all be worked around as well. As one S.O. poster said "Hackers are bored by this!" However, they are still very important and should be strictly checked. HartleySan also suggested running the original file name through a regex, and that is a solution which is strongly suggested by many sites. I found a good whitepaper about file upload security: https://www.owasp.org/index.php/Unrestricted_File_Upload A lot of the points are already covered in your book, but there are a few others also. Larry, if you or anyone else has any more input or ideas, please post them, as this issue is very serious and will help others in this forum. Matt

This might be of interest to many in this forum who want to do image uploads. I am working on a site where users can upload a profile image and I am essentially using Larry's image upload code as written. The site is responsive and has to allow image uploads from smart phones as well. The problem is that when I tried uploading images from my Android phone, I kept getting a "The uploaded file was not of the proper type." error. I then put in an echo statement to display the file type and MIME type and found out that the images had a type of "application/octet-stream" and a MIME of "image/jpeg", and since we are checking both the type and MIME type, that is why the upload failed. This is a major problem for smart phone users, as many Android devices set the type of images taken with the onboard camera to "application/octet-type". I know that application/octet-stream is a generic type and could represent any binary data, including a malicious file. However, the web is a constantly changing place, and we need to be flexible in order not to marginalize users. This is especially true in the case of smart phones, as they now represent a greater share of devices used to access the Internet than PCs. Telling a user that they can't upload an image because they have the wrong device is unacceptable! Is there a good solution that will allow greater flexibility in the types of images that can be uploaded, while still maintaining security? I found a solution on S.O. that might be acceptable: //if mime type is application/octet-stream (psp gives jpegs that type) try to find a more specific mime type $mimetype = strtolower(preg_replace("/^(.+?);.*$/", "\\1", $_FILES['form_field'] ['type'])); //reg exp copied from CIs Upload.php if($mimetype == 'application/octet-stream'){ $finfo = finfo_open(FILEINFO_MIME, '/usr/share/file/magic'); if($finfo){ $_FILES['form_field']['type'] = finfo_file($finfo, $_FILES['form_field']['tmp_name']); finfo_close($finfo); } else echo "finfo_open() returned false"); } It basically gets the MIME type and sets the file type to be the same. The problem is that the Fileinfo extension needs to be installed on the server. If anyone has any thoughts on this, it would be greatly appreciated! Thanks in advance, Matt

Larry, Thank you for the quick and decisive reply! I always trust your intuition on this sort of thing and I do agree that option 1 is the easiest to implement, query, and maintain (although I might not go as far as to say that it's "normalized", as multiple NULL values are being used to store data). That feedback is always helpful! Thanks again! Matt

Hello everyone, HartleySan and I are working on a web project for a university and my question is about the approach to take for the database design. Basically, the site is designed to help students find tutors for various classes. What happens is that a tutor fills out a profile with basic personal details, as well as price and availability. Students who register on the site can view the tutor's profiles, and when they find one they like, they can click on a "poke" button to send the tutor a request that they are interested in his/her services. Both students and tutors have a control panel where they can view requests they've made/received. After a student has requested a tutor, he/she will see the student added to a table in their control panel where they can either "accept" or "deny" the request. If they accept it, then the student is given a status of "pending" in both the student's and tutor's table. From there, the student can make the final "accept" to finish the transaction and then get the tutors personal contact information. I know that probably sounds confusing the way I described it, but it's really easy. The problem is how to model these 3 different transactions (student request of teacher, teacher acceptance of student, final accept by student) in database tables. There are several possible solutions, but all of them have positive and negative points: 1) Make one table with the student_id, tutor_id, date_requested, date_pending, and date_accepted. This would be the easiest and most secure, but the problem is that it's not normalized at all (not even 1NF). There would be empty columns when transactions aren't completed. 2) Make 2 tables: One with student_id, tutor_id, date_requested, and date_pending and another with student_id, tutor_id, and date_accepted. I like this one the most as it keeps the incomplete transaction in a separate table from the completed one (i.e. student finally accepts the tutor and becomes his customer) and is also more secure. 3) Make 3 tables (one for each step of the process). When a transaction occurs and a student/tutor go from one table to the next, they will not be deleted from the previous table, so that there is a record of all transactions. This would be the normalized approach, but the queries involved could get complicated. I know this post is getting long, so I'll end it here. If anyone has any input or advice as to how they would approach this, it would be greatly appreciated. Thanks in advance, Matt