[This post also appears on Medium] After spending long hours to have a fully working Circle CI / Elastic Beanstalk / Docker integration, I thought it would be useful to put my notes down so I will save some time next time I need to go through a similar setup and avoid spending long nights cursing the gods of the (internet) clouds.
I’ve recently had to develop a web app that shows Tweets locations on a map. It’s simple enough to extract a tweet’s location (when present), just check the API Docs for a Tweet object and you’ll find a coordinates field that reportedly:
I’ve spent countless hours trying to fix Django installations running on legacy Apache servers, these usually recurring every few months, a time span long enough to forget how the last fix was done. And for some reason, the docs are not mentioning this crucial features AT ALL! In the official Django + mod_wsgi documentation page, they don’t mention something so irrelevant such as THE MAIN DJANGO SETTINGS FILE?
tl;dr: just visit my Github repo for a simple boilerplate with basic documentation.
I have found 36,749 examples of boilerplates using a miscellanea of AngularJS, Gulp, Browserify, Grunt, RequireJS and all the usual suspects.
But none of them actually did what I needed, although I reckon it is quite simple: define Angular modules in separate files, so the code is nice and clean and all the various services, directives and controllers are pluggable without massive headaches.
At the same time, having completely ignored the existence of Grunt so far (thanks to frontend guys who took care of this), I thought it was a good time for a fresh start with the latest and more fashionables Gulp and Browserify.
I ended up creating my own boilerplate (or seed, if you like), it works defining angular modules:
I started playing with a Django-Python version of my current project and I needed a testing environment on my local Mac-based machine. The thing is I didn't want to use a dedicated MySQL server just for the Django deployment, as it seems really silly when I already have my quiet MAMP running in parallel with all the other CodeIgniter-PHP based projects.
It's harder than you may expect. I had to go through a painful series of steps to eventually have the whole thing working, and taking here some notes so I can have them handy when I'll need to do it again, hopefully it will help someone else. It still needs to have a "dedicated" copy of MySQL installed just to compile, but it won't be used to actually run the server.
step1. download and install the latest mysql community server from the DMG file, here. Then from the command line, add the fresh installation to the local path, this will be used to pick the sources needed for the python module compilation. Also add a variable for the dynamic library linkage, this will be used on execution of your python scripts to find the correct library:
step3. now we're ready to install django and mysql-python. the latter should compile with few minor warnings if and only if we have correctly executed step1 (just double check that paths in the export statements are correct)
if everything works fine, then the last command above should show the full path of the mysql module compiled during the installation process
step4. create a django project and configure the settings.py so as to use the MySQL server shipped with MAMP
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'django_test', # Or path to database file if using sqlite3.
'USER': 'root', # Not used with sqlite3.
'PASSWORD': 'root', # Not used with sqlite3.
'HOST': '/Applications/MAMP/tmp/mysql/mysql.sock', # Set to empty string for localhost. Not used with sqlite3.
'PORT': '', # Set to empty string for default. Not used with sqlite3.
}
}
take EXTRA CARE to double check that the host variable is set to MAMP's socket, hence /Applications/MAMP/tmp/mysql/mysql.sock
step5. in my case, it's really not happening frequently, I also had problems with my locale, as some of the programs I installed messed around with the environment, so I got the error "unknown locale: UTF-8" as soon as I tried to sync the django db. this is solved exporting the correct variables, THEN syncing the db
and this should be all! Just a final note: this post was assembled trying things and using different sources to solve each and every problem appeared along the road, these are the original URLs I had to collate to eventually see my working copy of django: dedicated mysqldynamic librarymamp socketunknown locale
It happened a million times: I had to check a regex in PHP and did not have a handy tool to see the result. I usually linger on Rubular, which is a great service but based on Ruby, and sometimes the results are a bit different [update 2015 - just use regex101, best tool around]. I decided to implement a simple regex checker similar to Rubular, but based on php preg_match.
So the little storm has arrived! I am now the proud (and tired) father of little Benjamin, and for this reason blocked at home with little or no time for doing anything but changing nappies and cooking super-proteic food for Val. But somehow I found some time amuse myself with a little piece of Processing and wrote a simple code to visualize tweets on a map of London during the Royal Wedding. Easy enough to foresee, tweets during the day are creating nice clusters around Buckingham Palace, Westminster Abbey and the super posh hotel where the Middleton's used to stay.
Here is the result:
To display this data I reused the information stored during the first phase of my Flux of MEME project, fetched from twitter with the Streaming API implementation in its Java flavour twitter4j. Processing is reading the information in XML directly from the database, hence a little PHP backend is providing the XML descriptor for all the posts locations.
Few days ago I had to add a wordpress installation within the same environment where a Codeigniter app was already running happily and undisturbed. It took me a while to figure out how to keep separate folders on the filesystem, and serve the blog from a subfolder of the main domain: it ended up that the solution is super simple, but apparently I am not the only one who had similar problems. Symptoms of a bad installation usually result in “no input file specified” messages or, even worse, downloading the php source code with all your precious database passwords shown in clear.
A year has passed since the beginning of the trial of Flux of MEME, the project I have presented during the Working Capital tour, and it is now time to analyze what has been learned and show what has been developed to conclude this R&D phase and deliver results to Telecom Italia.
the initial idea
It’s worthwhile giving a quick description of the context: Twitter is a company formed in 2006 which has received several rounds of funding by venture capitals over the past few years, this leading to today's valuation of $1.2B, still during the summer of 2009 the service was not yet mature and widespread as it may look now. At that time the development of the Twitter API had just started, this probably being one of the few sources, if not the only one, for geo-referenced data. The whole concept of communication in the form of public gossip, mediated by a channel that accepts 140 characters per message, was appearing in the world of social networks for the first time.
This lead to the base idea of crunching this data stream, which most importantly include the geographical source, then summarize the content, so as to analyze the space-time evolution of the concepts described and, ultimately, make a prediction of how they could migrate in space and time.
A practical use
It could allow you to control and curb the trend of potentially risky situations (such as social network analysis has been useful during the recent riots in London) or even define marketing strategies targeted to the local context.
The implementation
A consistent initial phase of research allowed to have an overview on different aspects: the ability to capture the information from Twitter, the structure of captured data, the ability to obtaining geo-located information, the classification of languages of the tweets, the enrichment of content through discovery of related information, the possible functions for spatial clustering, the algorithms for topic extraction, the definition of views useful for an operator and finally the ability to perform a trend analysis on the information extracted. All of this has resulted in a substantial amount of programming code, its outcome being a demonstrator for the validity of the initial theory.
space-time evolution of the concept "earthquake" in a limited subset of data captured during the period May 2011"
Few days ago I started the server setup for a web project @therumpusroom_ and, after receiving the traffic estimates, I thought a single Apache server was not enough to handle the expected load of visitors. For several reasons I want to avoid using a load balancer and multiple Apache instances, hence the decision to implement Nginx with MySql running on a separate dedicated server.
The whole infrastructure lives on Amazon Web Services and the web application - still under development - will rely on CodeIgniter. I have read quite a lot of articles on-line and stolen bits and pieces of configuration files, but none of them entirely reflected what I needed. I feel it is quite a common configuration hence I am writing down here the required steps and some code snippets, both for my personal records and also hoping it can be helpful for someone else with similar issues.
The premise: implement a CodeIgniter installation on Amazon EC2 with a dedicated DB server and content delivery network for rich media distribution.
Pre-requisites / specs: Ubuntu 11.10 Oneiric Ocelot 64bit with Nginx web server running on a large instance on Amazon EC2, dedicated MySQL server on Amazon RDS and Cloudfront CDN.
The steps:
1. choose your Ubuntu installation
I ended up choosing Oneiric Ocelot 64bit, I am always too tempted to try the latest, anyhow you can always find your own Ubuntu AMI using the super helpful AMI locator for EC2
2. start a basic NGINX installation
I used this guide on Linode to configure Nginx and PHP-FastCGI on Ubuntu 11.04 (Natty) as a starting point, just be aware of the following:
ignore the hostname configuration: it did not work for me and it is not relevant to make the web server work properly
start with the suggested config for nginx, but keep in mind you will need to finalize it later
also the init.d/php-fastcgi script in the Linode guide gave errors and was not working properly for me, so I have created a simpler version (you may need to manually create pid/socket folders before running the script the first time):
[caption id="" align="alignright" width="300" caption="Image via Wikipedia"][/caption]
Few days ago I had to deploy a simple wordpress installation for a project at @therumpusroom_ with my buddy @twentyrogersc. Super simple, yet it presented a little issue: having a personalized authentication form, displayed on a wordpress page, it was re-directing to the default login screen after specifying incorrect credentials. It came out that we needed to implement the whole authentication mechanism on a page, hence tweaking the simple post form to send data to itself instead of submitting to wp-login.php (which, btw is working fine if users provide the correct credentials, and also accept a redirect parameter to make users land on a pre-determined page).
Just for my records, I leave the code here, and I hope I could save some time to people who incur in the same problem. Here's my solution:
from the dashboard create a page and give it a meaningful permalink, such as log-in
create a php file and give it a name according to the wordpress convention: page-<permalink>.php (in my case: page-log-in.php)
write the php code to perform the whole authentication (hence including password check and signon), this is my code:
<?php
// if user is logged in, redirect whereever you want
if (is_user_logged_in()) {
header('Location: '.get_option('siteurl').'/how-to');
exit;
}
Eclipse is a great IDE, I don't know how could I live without it, but when it comes to PHP development, the PDT plugin shows some flaws unfortunately. It may happen that you need to declare an anonymous function and get a "compiler" error, as shown below:
just ignore it, the code is working perfectly well, so declaring an anonymous function will not give any problem, you just need to cope with the little red error message until the end of the project (or declare the callback as an identified function).
Below the code to declare the inner function for array_filter:
Over the past few days a couple of friends asked me more or less the same thing: retrieve a collection of Venture Firms and visualize a simple subset of important data, such as geo location, average investment, and short description. I have hence decided to spend a little time with some PHP/Javascript and implemented a simple application.
I started a personal project months ago, aimed at realizing a web and mobile infrastructure for wine tasting notes. During the same period I quit my old job, moved from my old flat in Florence to my wife's in London, found a new flat and a new job, moved again to the new flat, changed computer (the old one being back to the lab where I used to work) and started a couple of new projects, quoted in my previous posts.
I think it's enough to justify that my original project - for which I found what I think a cute name: "Cellarium" - is currently in stand by. Still I think it's a real shame, mainly for two reasons:
there are not official AIS and/or ONAV wine tasting applications for mobiles in the cloud (or at least I didn't found them)
being now in London, I must say that Italian wines are under estimated and are really hard to find (no, I'm not talking about the crap you usually pick up around)
If you are brave enough to read through the rest of the post, you will discover the following:
code is completely undocumented, shame on me, but I didn't have the time to do anything else, it's just there and you can use it at your own risk;
wines and wine tasting notes are stored in a sqlite database (stored in the mobile device) and I don't even remember its structure, the only way to retrieve how things are done is to take a look at the two inner classes defined inside CellariumProvider.java
saving and updating wines work fine, while wine notes still need a lot of coding, both from a data persistence perspective and interface implementation.
The application is intended to help while writing down wine tasting notes for both AIS and ONAV standards, as depicted in the following.
I'd be happy to help/contribute and finish the application, volunteers appreciated. If no one interested in this, I'll probably finish it when I'll retire, using Android v.714.23 on a nuclear-powered-artificial-intelligence-smartphone produced in Mars by Cyberdyne systems.
In the last few week I have been working on a project that involved the development of a particle animation system. The project is still under development and we are working under NDA: for this reason unfortunately I still cannot post any details about it, even though I will post details as soon as it goes live.
What I can definitely do is show the tests I had to develop using Flint particle animation system, developed by @Richard_Lord and give a brief explanation of the code, since I saw that a lot of people are having troubles with, to say the less, a little bit sparse documentation.
Let's show the results first, it is a tweaking of the logo tweener example found on the website.
It was not rocket science at all, since it was only matter of tweaking an already made example, still I noticed from the forum that loads of people got stuck while introducing some changes, and I had to go more in depth of some aspects related to EmitterEvents, ParticleEvents and Easing.
I wanted to use extensively the TweenToZone effect for a series of bitmaps which had to be displayed in the .swf, for this reason I decided it was better to create a class and reuse it when needed. The emitter can also reuse existing particles instead of blasting a complete new set of them, the code is the following.
public class Tweener extends Emitter2D
{
public function Tweener( init:Boolean, bitmapFrom:Bitmap, bitmapTo:Bitmap, particles:uint, lifetime:uint, color1:uint, color2:uint )
{
if( init ) {
counter = new Blast( particles );
addInitializer( new ColorInit( color1, color2 ) );
addInitializer( new Position( new BitmapDataZone( bitmapFrom.bitmapData, 0, 0 ) ) );
}
space-time evolution of the concept "earthquake" in a limited subset of data captured during the period May 2011"
