Speaker Info
- BSc (hons) Computer Science from De Montfort University, Leicester
- 3 Years working for Opera Software
- Mainly Spring MVC based web applications until widgets.opera.com
Introduction
- Opera Widgets are small web applications that run within the Opera
browser environment
- widgets.opera.com is the site where users go to discover new widgets
that are available for use
Project Background
- Needed more flexibility than the current site
- Old team busy enough with other Opera properties (my.opera.com, dev.opera.com, etc)
- We didn't want to learn perl
- But still wanted to keep open the possibility of passing the site back for maintenance
- Finally chose Pylons
Architecture
- http://www.theserverside.com/news/thread.tss?thread_id=48304
- Summary
- Two classes: One action coupled with the framework and One business logic NOT coupled with the framework
- Two classes: One action NOT coupled with the framework and One business logic NOT coupled with the framework
- One class: Action and business logic together in the same class coupled with the framework
- One class: Action and business logic together in the same class NOT coupled with the framework
Architecture - Spring MVC
- Spring MVC is type 1
- Controllers (actions) tied to framework
- Business logic encapsulated in Service layer and Domain Objects
Architecture - Pylons Demos
- Pylons examples are type 3
- Controllers (actions) tied to framework
- Business logic tied into the same controllers
def save(self, title):
page_q = Session.query(Page)
page = page_q.filter_by(title=title).first()
if not page:
page = model.Page()
page.title = title
page.content = request.POST.get('content','')
c.title = page.title
c.content = page.get_wiki_content()
c.message = 'Successfully saved'
Session.save_or_update(page)
Session.commit()
return render('/page.mako')
Architecture - Final
- Type 1 architecture
- Controllers (actions) tied to framework
- but only responsible for parsing input and populating tempates
- Business logic in service layer
class BugController(BaseController):
def new_action(self, id, version):
bug = Bug()
# Populate bug instance from FormEncode self.form_result
g.bugs_service.create_bug(bug)
# Redirect so the bug report can be viewed
return h.redirect_to('bug_detail_page', id = bug.id)
Tip - Cache Decorator
class WidgetsService(object):
"""
The widgets service class is the main entry point for interacting with
the store of available widgets.
"""
@cache(timeout=3600)
def find_categories_with_widgets(self, profile_id, browser_id):
"""
Gives the list of categories with an extra set attribute 'item_count'
which holds the number of widgets for the given profile and browser,
and only categories with item_count greater than 0 are returned.
"""
Tip - Cache Decorator
def cache(timeout=None, exclude_args=None, exclude_kwargs=None):
"""
General cache decorator that will cache the result of any function/method
using the module, function name, and call parameters as key. Everything
that is not None is cached.
"""
- Allows us to cache any method call in the application
- Very useful as there are many things we cannot cache on a request level
Experiences
- In general very happy with what Pylons has been able to provide, but
of course there have been challenges along the way
- Problems
- Sharing Controller Code
- Translation
- Genshi/FormEncode
- Debian Support
- Good Points
Problem - Sharing Controller Code
- Want an architecture that allows us to share controller code between sites
- Pylons has hard coded controller lookup path
- Current solution suboptimal
- Looking for better options
Problem - Genshi/Formencode
- Profiling shows Genshi taking 80% of response time
- Creating modular templates difficult to do nicely
- Strips or breaks FormEncode's <error:*> tags
- Moving to Jinja
- Still love Genshi for smaller tasks
Problem - Python 2.5 on Debian Etch
- Sysadmin mandate use of Debian Etch on servers
- Etch doesn't have binary modules for 2.5
- MySQLdb
- Python Imaging Library (PIL)
- Unstable built against newer libc6 so can't just pin the newer packages
- Repacking some modules in-house
OK - SQLAlchemy
- Project moving fast
- Not completley happy with it but not found a better solution
Good - Modular Architecture
- Easy to switch authentication modules
- Easy to switch templating language
- Easy to change session strategy
- Never has the framework stood in our way
Good - Performance
- Whole site runs on just 2 commodity blades for redundancy
- Apache 2
- Pylons
- MySQL
- MogileFS (tracker and storage daemons)
- Memcached
Good - Performance
- Record page hits last July ~400000 in 24 hours, with no problems
- (although I wouldn't like to see the result of one box going down in that period)
Summary/Feedback
- Implementing widgets.opera.com with Pylons has been fun and full of challenges
- All of which have been fixable
- There's still work plenty of work to be done
- Think you can do better?
- Want to work on similar sites?
