Notes from DrupalCon - Drupal 8 meets Symfony2

The following are my notes from Drupal 8 meets Symfony2 on Wednesday, March 21st, 2012 at DrupalCon Denver. The slides can be found at http://speakerdeck.com/u/fabpot/p/symfony2-meets-drupal-8

Symfony is...

• A reusable set of standalone, decoupled, and cohesive PHP components that solve common web development problems
• Best practices
• Borrows ideas from other frameworks and languages
• Flexible and extenible
• an Object-Oriented set of classes
• compatible with PHP 5.3 and later
• a full-stack web framework
  • Drupal is using some
• http://symfony.com
  • http://symfony.com/components
• https://github.com/symfony/symfony
  • https://github.com/symfony/XXXX
• irc.freenode.net/symfony

Which Open-Source projects are already using the Symfony Components?

Projects

• Behat - a BDD framework
• Doctrine - an Object Relational Mapper and Database Abstraction Layer
• Propel - an object relational mapper
• PHPUnit - a PHP unit testing framework
• Jackalope - a content repository

Frameworks

• Silex - a micro-framework
• PPI 2 - a framework

Products

• easybook - a book publishing platform
• Midgard CMS - a content management framework
• Zikula - an application framework
• phpBB - a bulletin board software

What about Drupal 8?

• ClassLoader
• HttpFoundation
  • HttpKernel
  • Routing
• EventDispatcher
  • DependencyInjection

Using the Symfony Components

• Acquiring:
  • Git
  • Archves
  • PEAR
  • Composer
• Symfony Components comes with Drupal 8

ClassLoader

• Loads classes on demand, only when needed
• Built to autload any PHP Class if it follows standard conventions
  • http://symfony.com/doc/current/components/class_loader.html

HttpFoundation

• Symfony leverages open standards, including HTTP (RFC 2616)
  • http://www.ietf.org/rfc/rfc2616.txt
• http://symfony.com/doc/current/components/http_foundation.html
• Defines a feature-full object-oriented layer for the HTTP Messages
• Replaces PHP native global variables and functions to allow writing better, more secure, and more testible code

Routing

• http://symfony.com/doc/current/components/routing.html

EventDispatcher

Similar to drupal hook system; biggest difference is that it’s OO.

HttpKernel

Implements dynamic part of HTTP specification.

HTTP Caching

• Symfony provides no caching layer for web pages; relies on HTTP cache headers
• Expiration
  • Cache-Control
  • Expires
• Validation
  • Last-Modified
  • If-Modified-Since
  • Etag
  • If-None-Match
• Expiration wins over Validation
  • Cache the page for a short period of time, 10 seconds for example, and every 10 seconds the page is validated
• Using the default HttpKernel allows Drupal to Benefit

Notes from DrupalCon - Keeping the lights on (operations and monitoring best practices)

The following are my notes from Keeping the lights on - operations and monitoring best practices on Wednesday, March 21st, 2012 at DrupalCon Denver.

“Measurement is the link between mathematics and science” - Brian Ellis, Cambridge, 1968

Primary topics

• Platform management, monitoring, and measurement
• Security testing and monitoring
• Monitoring - mean time to recovery is a key metric (how long does it take to fix)
• Ongoing operational security

Essential Monitoring Features

• Real-time AND trend monitoring
• Infrastructure based
• Custom plugin system
• Avoid proprietary languages to ensure anyone can contribute
• Runs your functional tests
• Active AND passive monitoring
• Push alerts
• Log analysis
• Escalation
• Quality of life - levels, rotations
• Remote command/”job” execution

Functional tests

• Use Selenium

Business metrics

• PageRank
• Things that are relative to the business
• Number of users

Technical monitoring

• Apc tool
• Service state
• Cron - execute from remote monitoring system like Nagios

Nagios Module

• http://drupal.org/project/nagios

Job Automation

• Jenkins is the defacto standard for continuous integration and deployment
• Codify and scripting all deployment activities

Logging

• Turn on syslog logging - instead of database, write to a text file
• Centralized off-server

Monitoring Overview

• Ping or HTTP result code alert monitoring || Live user story testing and trend analysis
• Crontabs and poormanscron || centralized cron management
• Logging to database only || Syslog logging to central host
• Logging in to see Drupal errors and available updates || Centralized Drupal monitoring
• Offsite backups || Off-cloud backups

Book recommendation

• The Visible Ops Handbook

Security Testing and Monitoring

• Tools and services to detect and respond to vulnerabilities and threats.

Detect

Finding the problem

Respond

• Mitigate, fix, alert
• Having a response plan before incidents occur

Vulnerabilities

• Weaknesses

Threats

• Ways to attack, whether or not they are succesful

Vulnerabilities (OAuth Top 10)

1. Injection
2. XSS - biggest problem in Drupal
3. Broken auth/session - using core? OK
4. Insecure direct object reference - manging access
5. CSRF
6. MIsconfiguration
7. Insecure cryptographic storage - site specific, SSH, using a VPN to encrypt traffic
1. Exception - password hash, encrypted information within site and database (encryption module)
8. Failure to restrict URL access
9. Insufficient transport layer protection - https
10. Unvalidated redirects and forwards

Detecting Vulnerabilities

• Automated code reviews
• Static: Coder Module Secure Code Review module, Acquia
• Dynamic: Not common
• Automated penetration testing
• Generic tools: Grendelscan (open sourcE), Fortify, Rational
• Drupal Tools: Acquia
• Manual code reviews
• db_query(“DELETE FROM {users} WHERE name = “ $name”);
• Manual penetration testing
• Be an intelligent robot
• Vuln.module (NEEDS PORT TO DRUPAL 7), Firefox: Tamperdata

Security review module

Responding to Vulnerabilities

Custom code:

1. Fix it
2. Test it
3. Deploy it
4. Contact customers (?)

Contributed Code

1. 4 steps above
2. Work out a simple, repeatable test case
3. Report the issue to the Drupal Secuyrity Team
4. Compare to http://drupal.org/security-advisory-poicy
5. Work with the Team and maintrainer to get a fix
6. something else???

Detecting threats

• Spam
• Can be obvious indicator, but only if you’re actually monitoring
• Defacement (can be hidden)
• Use version control, Hacked! module
• security_review.module
• Watch revisions
• Crowdsource (flag)
• Code injection (xss, php)
• IDS - PHPIDS, TinyIDS
• http://www.codetrax.org/projects/tinyids
• https://phpids.org/
• Web Application Firewall
• mod_security from apache
• Brute force password
• Read watchdog all day long
• Droptor http://drupal.org/project/droptor

Responding to threats

• Spam
• Mollom, Akismet
• Spam, flag_abuse
• Defacement
• Revert to good copies from version control
• Overwrite with new versions
• Node revisions, db backup
• Code injection
• Keep code safe
• Proactively block attackers at the firewall
• Brute force password
• login_security module
• Included in Drupal 7 core
• Help with everything: httpBL
• http://drupal.org/project/httpbl
• honeypot
• Reduces crawlers and malicious spiders

Site monitoring

• Internal/Free
• Views
• Mailmon - brand new
• Quant - charting
• Report - charting
• Chart (system_charts)
• External/Paid
• Acquia network - ~$350/year, includes library, support
• Acquia Insight
• Droptor - $24/month/site, monitoring only
• Drupalmonitor.com - unknown pricing

Three keys to ongoing operational security

• Vigilance
• Strong Chain
• Incident Handling

What are the things that we need to do after launch on an ongoing basis after launch?

• Maintain eternal vigilance
• Automate as much as possible
• Avoiding human error - often “I was too busy to get to it”
• Conduct periodic audits
• Never sleep

Periodic Audit Program

• Email Trent Hein of AppliedTrust for a copy of the PDF
• http://www.appliedtrust.com/trent

Avoiding weak links in the chain

• Education
• Training
• Awareness

Patching

• PCI DSS requires patching of all critical infrastructure within 30 days
• What:
• Linux or other underlying OS
• Firewall infrastructure
• Switches
• Wireless Access Points
• … more

Incident Management (needs to be written)

1. Initial Response
2. Notification and Escalation
1. Smallest possible group for as long as possible, then figure out communication
3. Response Strategy
1. Do we need to update? Notify users?

One important take-away

• Don’t use the same password on multiple sites you administer (Playstation Network)

Secure Site Admin Pledge

• I pledge to take the following steps to be a responsible Drupal site administrator:
• I have set a unique, strong password for any accounts with administrative privelegaes, and I do not share passwords across sites
• I use multi-factor authentications (e.g., ssh keys) for OS-level access and have password-only access disabled on my systems.
• I have and execute a patching plan that includes the OS, web server, and Drupal layers (including core, modules, and custom code)
• I have and execute at least a minimalist periodic audit plan
• I am aware of and comply with applicable information security requirements for the data that my site handles (HIPAA, PCI DSS, etc.)
• I monitor vulnerability announcement mailing lists for the technologies I use on my site
• I monitor my system regularly such that I know how it behaves under normal conditions
• I have a documented incident handling plan that I am familiar with and can use in an emergency
• I take responsibility for ensuring that any custom code is developed according to secure coding best practices and is evaluated before being put into production
• I will be eternally vigilant and investigate any unusual/suspicious site behavior
• I have a process in place to ensure non-production sites are appropriately protected from external/access /crawling
• I am an advocate for practical information security practices and avoid “Security theater” showmanship

Thank You!

Please get in touch to chat about these topics:

• Greg: greg.knaddison@acquia.com
• Ned: ned@appliedtrust.com
• Trent: trent@appliedtrust.com

Notes from DrupalCon - What's New in the Panels Universe

The following are my notes from What's New in the Panels Universe on Wednesday, March 21st, 2012 at DrupalCon Denver.

What IS the Panels Universe?

• Panels
• http://drupal.org/project/panels
• Layout manager
• May end up in core in D8
• Page Manager (CTools)
• http://drupal.org/project/ctools
• GUI for hook_menu
• Panels Everywhere
• http://drupal.org/project/panels_everywhere
• Replacement for blocks and templates
• A lot of work will be done on this over the next year
• Panelizer
• http://drupal.org/project/panelizer
• Panel nodes “on steroids”
• Fieldable Panel Panes
• http://drupal.org/project/fieldable_panels_panes
• Entities that can be used as fields
• Like theme module for blocks
• Field API Pane Editor (FAPE)
• http://drupal.org/project/fape
• Entity Revision Scheduling (incomplete)
• http://drupal.org/project/ers
• You’ll need to check it out of git; no releases yet
• Panopoly
• http://drupal.org/project/panopoly

New in Panels

• Redesigned IPE (In Place Editor)
• Can change layout
• Can change pane styles
• Less ugly
• Pane locking
• Can lock panes to a region or make them immobile
• Content people wanted to control where the content was; these ads and header elements can’t move
• Currently does not prevent deletion; will be added in the next couple weeks

Panelizer

• Like Panel nodes, but with context
• For any node
• Or user
• Or taxonomy term
• By bundle
• With defaults
• That can be picked by the node editor
• Very granular set of permissions
• Entity generic - plugins for users, for example
• Presents users with a slimmed down interface
• no need for the page manager UI
• Covers 90% of use cases of variants

Fieldable Panel Panes

• An entity that can be added as a pane
• Create on the fly or in an administrative UI
• Full access control
• Supports bundles through a hook (no UI)
• Like BEAN module for blocks
• Example: Create an entity with whatever fields you want, such as an image

Field API Pane Editor

• Form to edit a single field on a single entity
• Contextual link on a panel to hit that form in the overlay
• Offers ability to save revisions
• Isolation of experience
• Not tied to panels

Entity Revision Scheduler

• Create a “draft” revision that is not the normal revision used
• Schedule when a revision is set to the current revision
• Works for Nodes as well as Fieldable Panel Panes
• Entity agnostic
• Incomplete; more works
• Patch in the queue to unschedule

Panopoly

• Panels-based site-builder distribution
• Based upon OpenAcademy but generalized
• Still in its infancy
• Should provide a fantastic starting point for site building
• Now includes starter content

Dynamic subversion repositories with WebSVN

I am a careful man. For every development project, I have a corresponding Subversion repository that stores all assets relating to the project. For browsing (and blaming diffs), I use WebSVN to provide a friendly interface. I currently use DreamHost to host my repositories.

However, I got tired of updating the WebSVN configuration every time I added a repository. Therefore, I developed a short, dynamic method of just listing them all without need for reconfiguration. Add the following snippet to your config.php:

$repository_root = '/CHANGE/TO/YOUR/PATH/svn/';

foreach(glob($repository_root . '*', GLOB_ONLYDIR) as $repository_dir) {
  $repository = basename($repository_dir);
  $config->addRepository($repository, 'file://' . $repository_root . $repository);
  $config->useAuthenticationFile($repository_root . $repository . '.access', $repository, $repository);
}

Google Music - first impressions of the cloud-based music streaming service

I enjoy listening to music, and keeping my collection synchronized across computers and devices can be arduous. I am fastidious about tagging, using tools like TagScanner to author and normalize the tags, file names and folders. My playback and library needs are minimal; foobar2000 has served my needs nicely for close to a decade.

I wasn't impressed by Amazon Cloud Drive and Cloud Player; their overtly commercial approach (understandable in context), the clunky interface, and limitations of the service turned me off. Amazon has since offered unlimited storage for music if you subscribe at the $20/year level or higher.

Recently, I was invited to try the Google Music Beta, their new streaming music service. Given my previous experience, I was hesitant, but willing to give a competitor a try.

Client

The desktop client download and installation was fairly smooth, except you needed an application specific password. The client linked to the help page that contained in the middle of the content a link to generate that password, which was obtuse. Once I was beyond that, I used the wizard to select a folder to sync, and walked away for the night. The next morning, about 2/3 of the ~5GB folder I had selected had been uploaded.

The next step was to try the android app, which I downloaded from the Android Market. Overall, the interface is nicer than the stock app. I have not tried Bluetooth playback yet.

Android App

Music that had been uploaded was available to be streamed; I picked the first track of an album, and hit play. It churned for somewhere between five and ten seconds, then started playing. Sounded fine. Once you start playing a track, the player will download sequential tracks in the background to avoid this delay. Playback sounded fine.

I tagged and transferred music manually to my android device for a long time before I started using Google Music. When I installed Google Music on my desktop, it uploaded my collection. I installed the app on my device, which correctly listed all the local music along with all the music available to stream.

However, music that was already physically on my device was also listed as available to stream, resulting in two entries for every album and track. This is undesirable, and I shared my feedback with the Music Beta Team.

What happens to your music?

As an experiment, I removed one of the local albums, then downloaded it ("make it available offline") using the app. I used the Wi-Fi connection, and a ~75 MB album only took a couple minutes to transfer.

The music transferred to Android\data\com.google.android.music\cache\music with a numeric file name (ex: 534.mp3), a rewritten track number, and missing the embedded artwork, Album Artist, Total Tracks, Disc Number, Total Discs, and Publisher. The tag type was rewritten from id3v2.3|id3v1 to id3v1. The content looks untouched; same number of samples, bitrate, and so on, The artwork is stored separately, in cache\artwork. The file size was nearly identical, in proportion to the missing artwork and tags.

Both the cache and artwork folder have a .nomedia file (tells default apps like Gallery and Music to not index it).

I understand why they are normalizing tag data (save space - especially with album art, and excludes non-standard or unsupported fields), but it's irritating to have two different file versions.

It's not impossible to copy out the music from your android device once it's in there (if you want to copy an album from your device to another computer, for example). You cannot download the music from their web interface. They want you to use their application to listen to it, and they're avoiding the slippery slope of making it easy to duplicate your collection.

I have a guess about why they chose 20,000 songs as the limit; FAT32 directories can contain up to 65,536 entries, and each file/directory uses between 1 and 13 entries (depending on the length of the name), so that arbitrary song limit is well within the technical specification.

Overall impression

Google Music is a strong, interesting contender in the streaming music market; while it's certainly not the most open approach, I am liking Google Music more than the other music streaming alternatives that I'm aware of and it's quite mature and usable. That'll do, Pig. That'll do.

Turducken: Slaughter with Laughter

It's been a while. I switched from public education to a private company back in March and haven't had the time to write. I've gone through some life changes recently, and in short I will have more time to write for Free Software Magazine, compose technical articles on PHP development, and much more. I will back-date some content, mostly photography related. Thank you for sticking around, I appreciate it.

Turducken

I made a short, twisted video about what, why and how my family made a Turducken, a turkey stuffed with duck stuffed with chicken. A veritable avian Matryoshka tsunami of culinary urban legend, few have attempted to create this carnage loaf.

Syracuse St. Patrick's Day Photos

Photography

On Saturday, March 17th, I attended the St. Patrick's Day Parade in Syracuse. I was experimenting with the Nikon N70, and I did my best to put it to good use.

You can also view the set of 60 photos as a slideshow.

Consolidation

My apologies for my SAMPP entry; it was way, way too long. I've consolidated it on my web host using the dojo toolkit to make tabs. You can view the amended version at theconfluence.org/projects/sampp .