Emilie Posted July 23, 2014 Share Posted July 23, 2014 Hello, I'm new on this forum. I'm an amateur and intermediate web developer. English is not my mother tongue, so I apologize for all the mistakes I'll make in my messages! I've been reading chapter 12, 'PHP's Command-Line Interface' and have had no problem reproducing the different scripts. But now I'm trying to apply what I've learnt to other PHP scripts, and I've run into a fatal error: Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 70517676 bytes) in some_file.php on line 7 Line 7 is the following: $clean_file = str_replace('some_string', '', $file); I first tested my script on a small html file, so I know the script in itself works. The html file I'm now using is quite big (70 Mo)… and there's worse to come (300 Mo)! Should i modify the configuration file called php-config in /usr/bin in order to avoid this fatal error? If so, how can I change it? I tried to open it in my text editor, but it appears as a Unix executable file. Or is this one of the cases where I should change the php.ini behavior with the -d option? If so, what change should I make? With thanks for your help, Emilie Link to comment Share on other sites More sharing options...
Antonio Conte Posted July 24, 2014 Share Posted July 24, 2014 You need to alter the PHP configuration. The problem is that the script takes too much memory, and therefor gives you a fatal error. You can get your current limit by: php -i | grep memory Change it by: a.) Alter the php.ini configuration file. Up the declaration "memory limit" (and make sure it's uncommended.) b.) Or adjust it in a single script adding ini_set('memory_limit', '64M'); to the top your script. You can also use a flag to do this from the command line, but I don't remember that at the moment. I would recommend just changing php.ini. 1 Link to comment Share on other sites More sharing options...
Emilie Posted July 24, 2014 Author Share Posted July 24, 2014 Thank you very much for your answer. I feel very stupid, but where can I find the php.ini configuration file for PHP CLI? I know this version of PHP is in /usr/bin, but I see no php.ini file there: And the php-config file is a Unix executable. Link to comment Share on other sites More sharing options...
Larry Posted July 24, 2014 Share Posted July 24, 2014 Just to confirm, the script you're trying to use is 70 MEGABYTES? If so, you'll want to change your approach. Instead of loading the entire file into memory, I'd go through it a line at a time and do a search and replace on each line. Link to comment Share on other sites More sharing options...
Emilie Posted July 24, 2014 Author Share Posted July 24, 2014 Thank you for your answer, Larry. The PHP script is only 4 KB, but the HTML file is indeed 70 MB (it's a HTML file with lots of unnecessary styling inside). And I can't go through it a line at a time because it's one huge line! That's part of what I need to change. My text editor (BBEdit) managed to add new line breaks in a 300 MB file, but this 70 MB HTML file is buggy, so BBEdit cannot do anything with it. That's why I'm trying to find other ways of editing it. I can open it and read it without any problems in the command-line, I just need more memory to be able to add new line breaks and then delete all this unnecessary styling. Do you know where can i find the php.ini file for PHP CLI? Or where I could look for more information on PHP CLI more generally? I'll temporarily use ini_set as recommended by Antonio Conte, but as I have quite a few huge HTML files to edit, modifying the php.ini file could be useful. With thanks for your help, Emilie Link to comment Share on other sites More sharing options...
Antonio Conte Posted July 25, 2014 Share Posted July 25, 2014 This should get you the path php --ini 1 Link to comment Share on other sites More sharing options...
Emilie Posted July 25, 2014 Author Share Posted July 25, 2014 Thank you very much, Antonio Conte. You've answered all my questions. The php.ini file is in /etc, and I should have known, because I think Larry mentions it in his book. The script now works, but I had to allow 512 Mo memory. The default setting was 128 Mo, it wasn't enough; 256 Mo wasn't enough either. 512 Mo did the trick. Link to comment Share on other sites More sharing options...
Antonio Conte Posted July 25, 2014 Share Posted July 25, 2014 I work with EZ Publish which requires a lot of memory sometimes during installation due to legacy reasons. Therefor I had to do the same recently. Also, due to a recent composer bug, 1 GB of memory was required to update all dependencies on one of our projects. Crazy. Glad I could help. I agree with Larry that you should maybe look at solving the problem differently. Throwing memory on problems can help out, but it might not be suitable in the feature. Whatever works, I guess. Link to comment Share on other sites More sharing options...
Emilie Posted July 25, 2014 Author Share Posted July 25, 2014 I only need that much memory to start with because these HTML files have been compacted and made into 1 line only. Once I've introduced line breaks, I can use file() to create an array of lines, and then use a foreach loop. This 70 Mo HTML file is also buggy, which is probably why BBEdit can do nothing with it, since it previously managed to introduce line breaks into a 350 Mo HTML file. But it took something like 20 minutes, whereas it only takes 1 or 2 minutes from the command-line! Link to comment Share on other sites More sharing options...
HartleySan Posted August 3, 2014 Share Posted August 3, 2014 Just out of curiosity, where did this 70 MB HTML file come from, and what did you need to edit it for? It sounds insane. Thanks, and welcome to the forums. You've provided some great answers to other people's questions. Link to comment Share on other sites More sharing options...
Emilie Posted August 3, 2014 Author Share Posted August 3, 2014 Hello, HartleySan, This 70 Mb HTML file was probably created with Word or some other unsuitable application. Hence loads of unnecessary style attributes inside HTML elements, empty div elements, duplicate links (some appeared up to 20 times on one line). After "cleaning", it became approximately a 7 Mb file! Emilie Link to comment Share on other sites More sharing options...
HartleySan Posted August 3, 2014 Share Posted August 3, 2014 7 MB is still insane for an HTML file, but a good start, nonetheless. Thanks for sharing. Link to comment Share on other sites More sharing options...
Emilie Posted August 4, 2014 Author Share Posted August 4, 2014 Not all that insane since it represents a whole book and 7 MB was before minimizing the file. Look at the digital versions of computer books, and you'll see they are 10 to 50 MB. The HTML file is the largest file within an ebook format (ePub, mobi, etc.). Link to comment Share on other sites More sharing options...
HartleySan Posted August 4, 2014 Share Posted August 4, 2014 I totally believe that, but I would also think that for the HTML-format version of a book, you'd want to autogenerate the HTML page on the fly (e.g., pull some text with some form of markup from the DB) when the book was viewed from the web. Link to comment Share on other sites More sharing options...
Emilie Posted August 4, 2014 Author Share Posted August 4, 2014 I see what you mean. I hadn't thought of that, because I'm not concerned with the book being viewed from the web, at least for the time being. I only needed some means of opening those huge HTML files and editing them. I'm just an amateur programmer, so I first tried to open this 70 MB file with my usual text editor (BBEdit), and although I've been able to edit a 350 MB file with BBEdit (although very, very slowly), I couldn't open this 70 MB file with BBEdit. I was going through Larry Ullman's "Advanced PHP" book at the time, and through chapter 12 more specifically, so I just thought I would have a go with PHP CLI. It worked (with help from Antonio Conte!), and it is so much quicker than using a text editor that I now use it to edit huge files. But I'll keep in mind your idea of storing the text in a DB and generating the HTML on the fly if I ever need these books to be viewed from the web. Link to comment Share on other sites More sharing options...
Recommended Posts