How to make web2py faster

Avoid regenerating tables:

db = DAL('sqlite://storage.sqlite', migrate=runonce) 

Create indexes:

db.executesql('CREATE INDEX IF NOT EXISTS {table}_index ON {table} (id);'.format(table=table))

Compile byte code in admin

Serve static files direct from server

Move as much logic as possible from models into controllers and modules

Disable session if possible

Cache database queries and controller functions:
db().select(db.log.ALL, cache=(cache.ram, 60))


@cache(request.env.path_info, time_expire=5, cache_model=cache.ram)
def index():
pass

Let webfaction serve your static content

Create a static symbolic link application:
Name: web2py_static
App category: Symbolic link
App type: Symbolic link to static-only app
App doc: This creates a symlink from ~/webapps/ to the absolute path specified in the extra info field. 

Then in your website mount this at /static


Now requests to static files will be handled directly by webfaction and never reach your app.

web2py with uwsgi

I found Apache used too much memory so changed to uwsgi:

# build uwsgi
hg clone http://projects.unbit.it/hg/uwsgi
cd uwsgi
make -f Makefile.Py27
cd ..

# get web2py
wget http://www.web2py.com/examples/static/web2py_src.zip
unzip web2py_src.zip
cd web2py
# run web2py rocket server to setup password
python2.7 web2py.py
# set your password then kill the server
cp parameters_8080.py parameters_PORT.py
cd ..

# start uwsgi with 32 MB of memory, 24 threads, 1 process

./uwsgi/uwsgi --http=127.0.0.1:PORT --pythonpath /path/web2py --module wsgihandler -d /path/uwsgi.log -t 20 --async 24 --ugreen --limit-as 32 -r --no-orphans -M -p 1 --touch-reload /path/uwsgireload.txt --reload-on-rss 50

# reload server

touch /path/uwsgireload.txt

How to determine what Linux is installed

Kernel version:

$ uname -r
2.6.18-238.12.1.el5PAE


Distribution name:

$ cat /etc/issue
CentOS release 5.7 (Final)


32 or 64 bit:

$ getconf LONG_BIT
32

How to create an image mosaic

For a birthday present I generated an image mosaic using the metapixel command.
Firstly get atleast 1000 images in [base_image_folder] and sub-directories. Then process the data with:  


metapixel-prepare --width=64 --height=96 -r [base_image_folder] [tmp_thumbnails_dir]


This processing will take a while. Then generate the mosaic with:


metapixel -w 48 -h 64 -s 8 --metapixel [image_pattern_for_mosaic] [output_image_file] --library [tmp_thumbnails_dir]


This should only take a few minutes. For different resolutions change the size flag.

Automatically fix rotation of images

My camera stores the orientation of the images in the header, but this is not recognized by most programs.
To automatically fix the image rotations I use:

jhead -autorot

And to recursively rotate all images:

find . -name "*.JPG" -exec jhead -autorot -exonly {} \;

GoDaddy domains with webfaction hosting

In your GoDaddy account select Set Nameservers and then I have specific nameservers for my domains (Documented here)


Enter webfaction's domain name servers (Documented here):
ns1.webfaction.com
ns2.webfaction.com
ns3.webfaction.com
ns4.webfaction.com 


Wait an hour

Downloading data from GAE

As documented here you can download your data into a blob:

appcfg.py download_data --application=[app-id] --filename=[output dump file] --url=http://[app-id].appspot.com/_ah/remote_api

And then upload to another app:
appcfg.py upload_data --application=[app-id] --kind=[kind] --filename=[input dump file]


Or to work with structured data.

First generate the bulkloader file:

appcfg.py create_bulkloader_config --filename=bulkloader.yaml --url=http://[app-id].appspot.com/_ah/remote_api

Set the connectors to CSV

Download the data:

appcfg.py download_data --application=[app-id] --config_file=bulkloader.yaml --filename=[csv file] --db_filename=progress.sql --kind=[which entity]
--db_filename is important for large uploads to avoid re-uploading the same data

Deploying code with rsync

I use rsync to deploy code to my web server:

rsync --recursive --compress \
        --verbose --progress --stats \
        --times --perms --delete \
        --exclude "*~" --exclude "*.pyc"  \
        . login@server:path

This command will transfer the current directory and all sub-directories and files to my server, but only transfer modified files to save time and bandwidth.
verbose, progress, and stats display useful output
times / perms ensure the file timestamps / permissions are the same on my server
delete will delete remote files when deleted locally
exclude temporary files

Convert DVD to AVI

I tried a few programs to burn a DVD and found acidrip works best.

Remember:

• set size to around 1000 for each hour
• turn scale off
• detect the correct crop size
• preview results

Code syntax highlighting

I use this when I need to syntax highlight code snippets:
http://tohtml.com

Writing in Korean

Install ibus, ibus-m17n, ibus-hangul

Restart - I couldn't use ibus in Chrome or choose the Romaji keyboard option until restarting

Ctrl-Space to activate language. Change keyboard to Romaji for English keyboard.

Scim also has a Korean package but I couldn't get it to activate reliably.