Jump to content
Larry Ullman's Book Forums


  • Posts

  • Joined

  • Last visited

  • Days Won


Posts posted by ewbarnard

  1. Everything about Yii is so straight-forward that I'm surprised that access control continues to appear difficult and complicated to me. So far I've avoided any need to do any authentication or user login (because the sites are locked down to an internal network). That will soon change.


    I expect to need a number of user logins, with a small number of roles. I'll have 10-20 logins spread across 6-8 roles. It's very small scale in that sense.


    1. Does anyone have recommendations for an RBAC user management extension? I'd like the client to be able to manage users, and assignment of roles to users, without programmer intervention.


    2. It seems like I should be able to implement a simple strategy for tying either entire controllers or specific controller actions to those roles. I'm not sure what that strategy is, but I haven't tried to do this yet either. Practice makes perfect!

  2. Larry, agreed! I considered MySQL "load data" and indeed it does a screamingly fast import. Unfortunately I have not found any command-line way to distribute a row of data into normalized tables (address to the address table, product lookup-or-insert from the product table and insert its id, etc.). If that CAN be done, I'd love to be clued in! Perhaps a stored trigger on insert, but that begins to add another layer of complexity to what should be simple!


    My other issue is "scrubbing" the data. I detect various inconsistencies during import. That in itself is not justification for doing PHP-based import. It would make equal sense to to a fast command-line import and then do a second pass with PHP looking for those discrepancies. But, since I'm already in the PHP, I just do the discrepancy checking at the same time.

  3. Hi Larry,


    In your database chapter you have some tips on specific things which might be encountered. Here's a topic tip which for me was non-obvious.


    I need to import large chunks of data, meaning 10,000 - 100,000 rows of CSV data. For large data imports to MySQL InnoDB, I learned:


    1. Active Record is prohibitively slow. So is DAO, but not for the reason you'd think.


    2. My test platform performance (a mac laptop) bears absolutely no relation to production database server performance.


    Secret number 1 is that, for the MySQL InnoDB storage engine, you get massively better performance doing the entire import (hundreds of thousands of database queries) in a single transaction. InnoDB does some overhead things at the end of each transaction, and if you have a LOT of transactions, the number of transactions per second is extremely low. This behavior is distinctly different from MyISAM.


    This means that the whole point of transactions goes out the window. This scheme rolls back the entire import rather than rolling back a single row. But doing a transaction per row becomes prohibitively slow if you're doing a LOT of inserts at once.


    Secret number 2 is that "innodb_buffer_pool_size" needs to be increased from the default value, by a LOT. My own /etc/my.conf (dev platform) reads:


    max_allowed_packet = 12000000
    innodb_buffer_pool_size = 8G

    One symptom of the issue is when you truncate a table with several million rows and foreign key constraints. The truncate will take hours with the default pool setting. It takes a minute or two with a huge pool setting. Database performance on the inserts likewise goes way up.


    Because I am using a normalized database structure, each import row generates several database queries/inserts hitting the various related tables. That's why you would want to enclose all queries for a single row in a single transaction, and roll them back as a unit.


    What I did, instead, was bury all that inside a stored procedure. The stored procedure (defined as a Function) returns the main row insert id or null.


    Then, in php, I keep a count of "good" rows and "bad" rows. I use the Yii database log route for reporting the "bad" rows. If I logged them in the database during that transaction, and rolled back the transaction, I'd lose that logged information. Since Yii does not write the log messages until the end, the transaction is closed and I don't lose the Yii log. (Nice!)


    So, my method for doing a massive import of many rows of data:


    1. Adjust the MySQL Server setting for innod_buffer_pool_size.


    2. Enclose the entire import in a single transaction, or in as few transactions as possible.


    3. Run the entire import inside a try/catch block, exactly as The Yii Book demonstrates in the database chapter.


    4. Build the DAO command once, and bind the parameters for each row, as that row gets imported. In my case the row data are trusted, and the parameter binding is sufficient data validation.


    5. In my case, it's okay to "ride through" a failed import of a single row. Thus I hide the insert-error processing inside the stored procedure, and do NOT throw an exception.


    6. Any thrown exception triggers a transaction rollback, and if the import completes without throwing an exception, the transaction commits.


    7. I use the Yii log-to-database route for reporting import problems. They do not get written to the database until the batch process has ended, and therefore they do not get rolled back by a failed import.


    For the record, I drive all the above with PHPUnit. The Yii/PHPUnit integration is nice indeed! The batch import is implemented as a Yii console command, so PHPUnit invokes that with exec(), but each of the batch steps is a component which PHPUnit can exercise directly.


    In the same way, PHPUnit can directly exercise and validate the stored procedures by using Yii DAO inside the PHPUnit methods.

  4. Larry,


    Concerning Chapter 4, Generating Code with Gii (p. 69ff):


    In the tip at the end of the chapter (p. 76), you mention ways to customize Gii. This would be a great place to mention the Giix extension which you do describe later in your book.


    I use giix exclusively, and am extremely pleased. I find that I'm often tweaking table schema during development, and giix allows me to regenerate the base model without touching the model code that I write.


    Another item worth mentioning with respect to Giix: Here are a couple of low-impact ways to override the base model functions.


    Example 1. To add to the base rules, without copying the entire set of rules:


        public function rules() {        $rules = parent::rules();        $rules[] = array('filename', 'file',                          'allowEmpty' => false,                          'types' => 'csv',                         'wrongType' => 'The load file must be .csv',                         );        return $rules;    }

    With the above, you can regenerate the base model class with giix, and not have to re-edit your rules() method.


    Example 2. Tweak one of the attribute labels. Again, this allows you to tweak a single label without copying the entire attributeLabels() method, and therefore not having to edit anything if you re-generate the base model class:


        public function attributeLabels() {        $labels = parent::attributeLabels();        $labels['filename'] = Yii::t('app', 'Load File');        return $labels;    }


  5. I have been using the giix extension for a couple of months now. I have been making a lot of table schema changes during development, and the fact that giix splits the model into a base part and a localized part is extremely helpful. When the model changes, such as adding a foreign key pointing to that table, rebuild the base model without touching your code.


    Be VERY careful when re-generating model or CRUD code! Be sure you preview everything, item by item, to ensure you don't overwrite something you meant to keep. In the views, I often copy-and-paste from the "diff" preview into my existing edited file to pick up the new or changed field/relationship.


    Ed Barnard

  6. I've been a full time PHP developer for several years, and doing software development for several decades. I began evaluating Yii as a development platform four months ago. I've now been writing PHP with Yii exclusively for a couple of months.

    I found the Yii learning curve to be surprisingly difficult. What made the difference for me? Larry Ullman’s book.

    I first read “Web Application Development with Yii and PHP” by Jeffrey Winesett. It was a great start, but I found I still was not able to get my first application completely put together.

    I then read much of “Yii Application Development Cookbook Second Edition” by Alexander Makarov. That helped with specifics, but I still was not able to just sit down and “do” my next application.

    “Yii Rapid Application Development Hotshot” by O’Meara and Hamilton III helped with some good ideas, but still didn’t take me in the direction I needed.

    I’m happy to state that “The Yii Book” by Larry Ullman got me to that point where I can sit down and write the PHP/MySQL application I need to write.

    It’s exactly as Larry explains in “The Goals for This Book.” What I needed was the big picture, “why you do things in certain ways.” I already know what I want to accomplish in the way of code and design; I just don’t know how to go about accomplishing that task using Yii.

    Larry’s book got me over that hurdle. I’ve made a lot of use of the Yii forums and tutorials. The help is there!

    I’ve found my share of “gotchas.” For example the gii/giix code generator does not create the right class names when your MySQL table has underscores in the name. The code generator uses CamelCase instead of underscores. Meanwhile, Mac has gone from case-sensitive to case-insensitive filesystems by default. Your code works fine on the mac, and when you move it to the production server running a case-sensitive Linux filesystem, your class files cannot be found.

    I find the necessary clue to the problem in the Yii forums, and was able to fix my copy of the code generator. Now I just have to figure out how to contribute the fix back to the community!

    Different people have different learning styles. I best learn from books. Your own results will vary. But for me, Larry’s book is just what I needed to get me past the “tipping point” and Just Do It, whatever I need to get done using the Yii framework.

    I’m about half way through the new Chapter 19. I’ve been writing my own components, and the timing of Chapter 19’s arrival is perfect. I have a whole list of things which I “almost” got right! I can easily see how Larry’s recommendations make good sense, and how I can improve how I do those areas of code.


    Ed Barnard
    Cannon Falls, Minnesota

  • Create New...