Saturday, July 27, 2013
I updated the script with feedback I got from the blog and reddit and fixed tab completion to work correctly on a mac by default.
Friday, July 26, 2013
Tuesday, July 23, 2013
Its a little rough I admit, and I welcome any feedback for improvements. As a result, I'm now able to run commands like the following anywhere once I have my virtual environment loaded.
(spock)jbisbee@tacquito:~/src/spock$ manage dbshell Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 95 Server version: 5.5.31-0ubuntu0.12.04.2 (Ubuntu) Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
Edit: I've gotten some feedback and I realize I didn't sell the real reason behind dynamically setting the DJANGO_SETTINGS_MODULE variable. I use Django at work and we have moved from the generic one file settings.py to a settings directory which looks something like this
settings/ base.py dev.py # imports from base.py test.py # imports from base.py production.py # imports from base.py jbisbee.py # imports from dev.py
This way you can have cascading Django settings with overrides at multiple levels. This bash function I wrote finds the settings directory and will set DJANGO_SETTINGS_MODULE to the user's specific dev settings (or dev if the user can't be found)
Disclaimer: If you manage your Django project's settings differently let me know. I'd love to make the determine_django_module_settings more flexible and patches are more than welcome!
Saturday, July 13, 2013
created = models.DateTimeField(auto_now_add=True) modified = models.DateTimeField(auto_now=True)
The problem I ran into was that I wanted to keep the legacy timestamp from the old database when I loaded the record for the first time then have the auto_now kick in. The problem is that auto_now is very stupid (as it should be) and doesn't understand the concept of insert or update and whether your new record is coming in with pre-populated DateTimeField attribute. It gladly throws away any value away and always replaces the value with the current time. I didn't notice it on first import but soon realized auto_now wouldn't meet my needs.
After a quick search, I found this Stack Overflow question that had the perfect fix for my problem. It suggested implementing my own save that would allow me to pass in values for created or modified but would fall back to the current time. I changed the code a bit because I didn't need a created timestamp for my model, but this should give you an idea.
Note: you have to remove any additional values from kwargs before passing to the parent class or super.save will complain that created and modified aren't valid arguments.
import datetime class User(models.Model): created = models.DateTimeField(editable=False) modified = models.DateTimeField() def save(self, *args, **kwargs): # original Code: called datetime.datetime.today() # multiple places. Modified kwargs # # created = kwargs.pop('created', None) # if created: # self.created = created # elif not self.id: # self.created = datetime.datetime.today() # # modified = kwargs.pop('modified', None) # if modified: # self.modified = modified # else: # self.modified = datetime.datetime.today() # Edit #1: Changed example to more efficient # version from blog comments. # single today() # # today = datetime.datetime.today() # # if not self.id # self.created = kwargs.pop('created', today) # elif kwargs.get('created', None) # self.created = kwargs.pop('created') # # self.modified = kwargs.pop('modified', today) # Edit #2: no longer pass arguments via save # was orignially how I wanted to solve # the problem and how I felt auto_now # should have worked. Thanks to # indosauros from reddit. :) # # On inital creation of the object # only set today if values are not # already populated. auto_now* # wipes them out regardless today = datetime.datetime.today() if not self.id: if not self.created: self.created = today if not self.modified: self.modified = today else: self.modified = today super(User, self).save(*args, **kwargs)
If you see any glaring python or style related issues, please let me know. I'm still finding my way pythonwise and would love any best practices feedback.
Monday, July 8, 2013
Edit: Turns out utf8 and utf8_general_ci is not the full utf8 implmentation in mysql. See this blog post for an in-depth explaination on how they're different (has to do with 4 byte utf8 characters being thrown out by mysql)
Edit #2: From reddit, credomane stated that character-set-handshake caused him troubles. With existing applications. I'm going to do a bit more research, but here is his response to sharing this blog entry on r/mysql
You might want to consider removing character-set-client-handshake or at least mention it could cause troubles with some programs that use mysql. Changing it to false or skipping the handshake makes MySQL behave like MySQL 4.0 according to the mysql docs. When I was looking over your blog post yesterday and checking the mysql docs that seemed like a bad idea to deny the client the ability to change to a character set it prefers/designed for.
I went through the whole process of taking mysql offline, backing up my databases, altering/converting the tables, updating the mysql config and bringing mysql back online. At first everything seemed fine then I started to notice a lot of 500 errors appearing in my apache2 access logs and some things not working. Mainly applications sitting behind apache2's mod_wsgi. After spending several hours restoring databases and redoing everything database by database I eventually found I could convert all my databases just fine. Know I know the problem isn't in the database conversions but with my config changes.
character-set-server = utf8mb4 collation-server = utf8mb4_unicode_ciCaused no problems what so ever but as soon as I addedcharacter-set-client-handshake = FALSEI started having problems again. Investigating the problem applications show that they desire utf8_unicode_ci but since the mysql server says utf8mb4_unicode_ci only, they abort the mysql connection when they can't get the charset they wanted to prevent any possible corruption.
collation-server = utf8_general_ci init-connect='SET NAMES utf8' character-set-server = utf8character-set-client-handshake = FALSE character-set-server = utf8mb4 collation-server = utf8mb4_unicode_ci
Mysql reference: character_set_server, character-set-client-handshake, collation-server,
Special thanks to johns-appendix from my reddit post to r/programming for pointing out that I had the wrong encoding. I welcome the down votes as I learned something. :)
Thursday, July 4, 2013
About 8 months ago I went to Amazon.com to rent a streaming movie and saw an ad for the The Nest Learning Thermostat, saying it was one of the hottest selling items on Amazon. I was a bit skeptical because I figured a thermostat is a thermostat right, and asked myself, "why is this thing so popular?"
Well it turns out that the magic is in the merging of technologies. The Nest combines
- A Thermostat
- A motion sensor
- Data from the internet
- Great interface (like the interface on an iPod)
The real magic is in the motion sensor. The biggest problem with optimizing a thermostat is you have to sit down and program the schedule you want it to follow. The Nest takes care of that for you. About a week after I installed it, I was walking down my hallway and the Nest lit up and stated it had, "learned my schedule" and would optimize my cooling. I was a bit taken aback but it makes perfect sense. The best part is that the default behavior is after it goes into "away" mode it doesn't turn back on until it senses motion again.
The Nest also does something else very subtly, it motivates you slowly turn up your target temperature to earn a "green leaf". I live in Florida and used to run my thermostat at a steady 77 degrees. Now with the Nest, I've bumped that up to 80 and supplemented with fans and surprisingly haven't noticed much of a difference in temperature. What I have noticed is a drastic change in my power bill.
Disclaimer: I work during the day and have no one at home that would set off the motion sensor to turn the air back on. If you do have someone at home all day or you're one of the awesome people who actually program their thermostat, then most likely you won't see savings as large as mine. Also I live in south Florida were air conditioning is king and we only run the heat one or two times a year :)
Wednesday, July 3, 2013
I then briefly considered creating a bash function that would determine manage.py's location on every invocation but figured that was silly and decided I might as leverage virtualenv's virtual environment and as Ron Popiel says, "You set it and forget it!"
It only took a minute of poking to find the postactivate hook file within my ~/.virtualenvs directory. Below is the fruit of my labor.
#!/bin/bash # ~/.virtualenvs/postactive # This hook is run after every virtualenv is activated. VIRTUAL_ENV_NAME=$(basename $VIRTUAL_ENV) SRC_DIR="$HOME/src" if [ -d "$SRC_DIR/$VIRTUAL_ENV_NAME" ] then MANAGE_PY=$(find "$SRC_DIR/$VIRTUAL_ENV_NAME" -name "manage.py" -type f) if [ -e "$MANAGE_PY" ] then alias django="python $MANAGE_PY" else unalias django fi else unalias django fi
The end result is that I have a new django alias that will work anywhere and act as though I typed python manage.py
jbisbee@beni:~$ workon django-tutorial (django-tutorial)jbisbee@beni:~$ alias | grep django alias django='python /Users/jbisbee/src/django-tutorial/mysite/manage.py' (django-tutorial)jbisbee@beni:~$
Disclaimer: I'm making a big assumption here that the virtualenv name you're using is identical to the name of the project you're working on. It's a pretty big assumption so I apologize if it does't work right out of the box for you.
Tuesday, July 2, 2013
+Marlon Bailey , looking at you buddy!
1. Install python and python-setuptools
You'll need to install python and python-setuptools before you begin. Macs come with these packages pre-installed so you can skip this step if you'd like.
2. easy_install virtualenvwrapper
Then you'll want to install virtualenvwrapper via setuptool's easy_install. It should be the only package you should ever install with easy_install.
jbisbee@benny:~$ easy_install virtualenvwrapper Searching for virtualenvwrapper Reading http://pypi.python.org/simple/virtualenvwrapper/ Best match: virtualenvwrapper 4.0 ... Finished processing dependencies for virtualenvwrapper
3. Find locaiton of virtualenvwrapper.sh
Once you have virtualenvwrapper installed you'll want to see where it was installed in your path. Please note that it is a shell script and does end with the .sh file extension.
jbisbee@benny:~$ which virtualenvwrapper.sh /usr/bin/virtualenvwrapper.sh
4. Add virtualenvwrapper.sh to Your Shell Startup
# your ~/.bashrc or ~/.profile file export WORKON_HOME=~/.virtualenvs # virtualenvwrapper.sh may be installed in # /usr/local/bin on your system source /usr/bin/virtualenvwrapper.sh
5. Reload Your Shell
jbisbee@benny:~$ source ~/.bashrc # source .profile
6. Create a New virtualenv Environment
Now you're in business to start creating virtual workspaces and installing libraries within them with pip. Using django as an example :)
jbisbee@benny:~$ mkvirtualenv tutorial New python executable in tutorial/bin/python2.7 Also creating executable in tutorial/bin/python Installing setuptools............done. Installing pip...............done. (tutorial)jbisbee@benny:~$
7. Install Python Modules with pip
Now install python modules with pip within your virtualenv sandbox:
(tutorial)jbisbee@benny:~$ pip install django Downloading/unpacking django Downloading Django-1.5.1.tar.gz (8.0MB): 8.0MB downloaded Running setup.py egg_info for package django ... Successfully installed django Cleaning up... (tutorial)jbisbee@benny:~$
Have fun and let me know if you run into any issues as I wrote this post pretty quickly and may have left something out.
First you'll need to add a line to your .bashrc or .profile setting your PYTHONSTARTUP environment variable
# ~/.bashrc export PYTHONSTARTUP=$HOME/.pythonstartup.py
Then copy this scripts contents to file your new environment variable points to
# ~/.pythonstartup.py try: import readline import rlcompleter import atexit import os except ImportError: print "Python shell enhancement modules not available." else: histfile = os.path.join(os.environ["HOME"], ".pythonhistory") import rlcompleter readline.parse_and_bind("tab: complete") if os.path.isfile(histfile): readline.read_history_file(histfile) atexit.register(readline.write_history_file, histfile) del os, histfile, readline, rlcompleter, atexit print "Python shell history and tab completion are enabled."
That's it. Now anytime you pop into the python shell you'll have history from previous sessions as well as tab completion. Enjoy!
Edit: If you're not running the latest version of OSX you may need to change the readline.pasre_and_bind line to the following
#readline.parse_and_bind("tab: complete") readline.parse_and_bind("bind ^I rl_complete")
Now anytime I pip install anything the package gets cached locally so I won't have to fetch it again.
CREATE DATABASE database_name; CREATE USER 'who'@'localhost' IDENTIFIED BY 'password'; GRANT ALL ON database_name.* TO 'who'@'localhost';
Now this is really basic stuff and I realize it, but the point is that I hate looking it up. The sad thing is that after writing this blog post I'll probably never forget it.
Monday, July 1, 2013
I recently took the job as AlertSite's Architect and I'll be switching gears and beginning to code python. I plan on keeping this blog pretty tech related, I plan on slipping in few life hacks as well has some light Team Fortress 2 banter as its one of my guilty pleasures. I've tried blogging in the past with minimal success, but for some reason I think this time it's going to stick... We'll see!