Enterprise Drupal @ SF State

Choosing Drupal
Began with search for WCM for DoIT
Tested many (Plone, OpenCMS, Joomla, etc.)
Settled on Drupal for functionality and a11y
Started w/ 1 virtual server, no real environment

Started with 5.x and moved to 6.x

Campus already had some Drupal exposure

Ancient History
Portals required on CSU campuses

Project Approach (IBM

Websphere)

Collaboration between SFSU Team

and IBM Team

Approach according to the

established IBM Method

An emerging consensus regarding

portal development includes the
following major best practices and
considerations:

There should be one AND ONLY

ONE horizontal portal on campus;

Portals should be developed

What happened previously

IBM Portal took years to implement base

features. Required constant maintenance
and time-consuming PMRs.

Other groups/depts. Didn't fully buy-in,

saw IBM WebSphere as a black box

Project Manager/Maintainer left

Said hey How about going to Drupal,

they have it running successfully for
WCM, would maybe take a couple months
to switch

IBM Lotus WCM vs. Drupal

Java

JDK/JavaEE/JSR
168/WSRP

PHP

drupal_hook API

Services module

Portlet Factory

Any IDE/Editor

Monolithic

Broke weekly

Core is relatively
small

Few expertise

Huge community

No community

Only support is
PMRs

Support via
forums, IRC,
Acquia

The A Team

Me Architect /
Developer / Themer

Supakit superkid
Kiatrungrit Developer

Bora Kou Systems

Admin
(Part time on this
project)

Jason D'Silva
Student Asst. Developer (10
hours/week)

Management position
was actually vacant
during our 2.5 month
sprint. So had a parttime interim project
manager.

Scrum style agile development

What we liked:

What we didn't like:

Quick
turnaround

Scope creep

Weekly
feedback

Cramming at end

Loss of resources

Clear deadlines

Not enough time

for testing/QA

Daily checkins

Working a lot

Learning a lot

Demo

[demo] of SF State Drupal Portal

http://inside.sfsu.edu/portal
http://drupal.sfsu.edu/drupal

Architecture

2 Dedicated RHEL 5 boxes run Apache (Xeon 2 ghz + 4 GB RAM)

2 Dedicated RHEL 5 boxes run MySQL (Xeon 2 ghz + 4 GB RAM)

1 Dedicated box runs MySQL cluster mgmt node (+ MMM

replication)

Load balancer in front + 2 mysql db connection scripts

Big IP Load Balancer

MySQL DB
connection test
script

Apache+PHP 5.2

Apache+PHP 5.2

MySQL Cluster 7.x

MySQL DB
connection test
script

MySQL Cluster 7.x

MySQL
Mgmt Node

Drupal & MySQL Cluster :(

Drupal will not run out of the box on ndbcluster tables
Problem: menu_router table and locale_sources tables
A bug was filed with MySQL, rejected because ndbcluster
doesnt support row lengths of that size
Issues on d.o. filed to address this in core
http://drupal.org/node/391130 and http://drupal.org/node/703916
Split the menu_router table into smaller, more manageable tables
Remove the key reference on locale_sources or change the key
data type to be something other than text/blob
Workaround at http://www.coderintherye.com/

SF State Custom Modules

User Interface Modules

Utility Modules

Address change

connectioncheck

Benefits pps

deletecookies

Distribution Groups management

encryption

Emergency contacts pps

langhide

Ethnicity resurvey

Scheduled message

Financial Aid Offer Letter

Show more filter

Financialaidofferletter admin

Shibboleth data mapper

Military status

sfsulogout

Name Change Self Service

shibbolethdata

Nextmuni

validator

Official Transcript Request

xmlservice

Getting the Campus Community

to Develop

Offered two trainings

1) How to install Drupal on your localhost

2) How to create Drupal modules

3) (Future) How to create web services

Need hand-holding

Monthly meetings
and code workshops

How to get a Module (portlet) into Portal

First content should be discussed with the oversight group
before beginning development to ensure Portal is the
correct place for it
Some departments or groups may have a page in Portal
that they are responsible for and control the content for
All changes must go through the development cycle
including unit and load testing
Security testing of code (Plug for Cracking Drupal)
Clean up and review of code

Development cycle
Develop code
Run unit tests
Check into SVN

Additional development
Interface testing
Tag in SVN

Verify automated tests

User acceptance testing
Content is added directly here
Management approval

Automated tests run

If any tests fail, there is no push to
production

Automated tests
If any tests fail, system is rolled back to
previous build

Prepare
Define requirements including functionality and UI
Remember you are working within the Portal template so many UI elements are
predefined

Define interfaces to backend systems

The most common interfaces are

REST/ WebServices that should follow w3 standards

LDAP (V3)

Determine where portlet will be located and who should have access

Local development
Build portlet
Run unit tests
Unit tests should include all UI components and all backend interfaces

Check into svn

Request dev build

Backend
Work with the data and application owners of the source systems to
build the required backend interfaces

Make sure you follow the defined security model if there is one, or
work with us to define one if the system you are connecting to has no
model defined

Set up web services

Development (DEV)
Run unit tests
Run integration tests, finalize development
Peer review all code
Tag SVN code for test
Request Test build
Code is pulled from SVN into Dev

Test/ Quality Assurance (TEST)

Define test plan and execute all functional tests
Tests must include all functions, UI components, and backend interfaces
Perform user acceptance testing
Perform load/performance testing
Submit automated test suite for Staging and Prod
This test suite must be comprehensive

Make any content only changes directly in Test

A manager must sign off indicating the following:
He/she is aware of all changes going into production and approves them for the
specified date
The automated tests sufficiently cover all functionality
All code has been peer reviewed
The load/performance tests prove that the application meets the minimum requirements
Tag code in SVN for roll out

Code is automatically pulled from TEST at 1am and deployed into staging

Staging
Automated test suite is run
If any tests fail, production roll out is not performed
If all tests pass, promotion to Prod occurs automatically at 6am

Production (PROD)
Automated tests are run
If any tests fail, production is rolled back to previous release

Accessibility

508 compliant

All pages were run through manual checks

Theme is color contrast compliant

Testing out using the accessible_content

module developed at CSUMB

Security

IP based Access for back-end services

Everything is behind a firewall

Everything is behind Shibboleth auth

Core PHP Module is removed

Role based access to pages, panels

Daily log checks (in future, archive logs in

central change mgmt repository)

Grendel-Scan

OWASP standards

Scalability

We are not very big

Handled ~500 concurrent users without
problem

Peak ~5,600 visitors/day

357,000 visits and 1.7 million page views

from March-May
Shibboleth single-sign on & Google
Analytics

Servers are way underutilized, use about

15% cpu on norm

Performance

Initial Build took 7.5+ sec. Load times

Used Xdebug to find drupal_http_request

was problem. Used cron to cache

Down to 3.5 sec load times.

Installed APC via PECL (quite easy)

(Make sure to give it enough memory)

Down to ~1.5-2 sec load times.

Ensure Oracle db connects quit if can't

reach (need to upgrade to 11g)

Where we are Now vs. Where we

want to be

Now campus websites

are fragmented,
outdated
Not all sites use Univ.
Template, and some
don't use it correctly or
update it timely

Depts. want to put sites

in centrally managed
highly available Drupal
install for easily
keeping sites up to
date

Depts want to move to

central hosting

1 Drupal theme means a

consistently updated
look

We have a monthly
meeting Drupal User
Group

Need installation profile

for point & click

Tools to consider

Aegir (still maturing when we looked at)

Open Atrium

Pressflow

Memcached/Boost

Cloud hosting via Acquia/AWS or

Rackspace as well as consultants

Drupal is not the solution for

everything (though I love it)

Drupal is not

our document management system

(though it could be someday) We use
Sharepoint

our project management system (again

Sharepoint)

our help desk ticketing sys (remedy, gia)

It can be made to do all these things, but

other products are out there

The Importance of Community

Drupal.org

Bay Area / Berkeley / South Bay DUGs

IRC: #drupal-support or #drupal

On campus: http://drupal.sfsu.edu/drupal

Monthly Drupal meetups (now workshops)

California Higher Education Group:

http://groups.drupal.org/california-highered

California Higher Ed Drupal Sites

California Institute of the Arts

CSU Monterey Bay portal (login

required)

CSU Monterey Bay Catalog

Moss Landing Marine Laboratories

San Francisco International Airport

(SFO)

SF State University Portal (login

required)

SF State Drupal Sites (login

required)

Stanford University Sites using

Drupal

UC Berkeley Drupal
Sites
UCLA Graduate
Students
Association
UCLA Semel Institute
UCLA Family
Commons

UCLA School of
Public Affairs

Questions?