Twitter-Eggs

Twitter IDs approaching 64 bit

Oho, Twitters internal IDs are moving towards 64-bit integers. So remember to update your e.g. MySQL integer columns to BIGINT.

via dev.twitter.com:

This means that if you currently use signed 32-bit integers to track user IDs in your system, you should update your code to use 64-bit integers as soon as you can. To be safe, and to provide a date to aim for, we recommend making this change by July 1, 2013.

Completely remove MySQL server including all the data from Debian

I just tried to uninstall MySQL on Debian via

apt-get remove --purge mysql-server

apt-get remove --purge mysql-client

apt-get autoremove

and wondered that the directories /var/lib/mysql /etc/mysql still existed after the commands. Then I recognized, that a

apt-get remove --purge 'mysql-server.*'

is doing better and removes everything.

Note:

if you get a

/etc/init.d/mysql: WARNING: /etc/mysql/my.cnf cannot be read.

error when you try to re-install MySQL, try a

apt-get purge mysql-common

and try to re-install again.

uninitialized constant MysqlCompat::MysqlRes

I just had the following problem when trying to start a Ruby on Rails application on a fresh installation of Mac OS X Snow Leopard after I installed the mysql gem via gem install mysql:

uninitialized constant MysqlCompat::MysqlRes

I searched a bit and found a solution here. It was a problem with a buggy version of  the Mysql driver. So you need to use a different version when installing the mysql gem. So first uninstall the existing gem via

gem uninstall mysql

and reinstall it via

export ARCHFLAGS="-arch i386 -arch x86_64" ;sudo gem install --no-rdoc --no-ri  -v=2.7 mysql -- --with-mysql-dir=/usr/local/mysql  --with-mysql-config=/usr/local/mysql/bin/mysql_config

Twitter: Using file based message queue instead of SQL database

Just found this in a pretty old article about Twitter and it’s architecture.

Now, a more sane way to do this (if I may be so bold) is to keep a message queue for every member, and whenever someone posts a new message, copy a pointer to that message into the queue of every member who’s subscribed. Then when someone polls, you just have to get the messages out of their queue and send them along. If that sounds kind of familiar, it’s because that’s exactly how email works — the Twitter posting interface is like a listserv, and the polling interface is like a POP server.

The cool thing is that you don’t even need a database to do this. You can store all of this stuff using very simple file formats, with one file per message queue. Since this is so much simpler than triple-decker SQL joins, one server can handle more requests; and if that’s too much, you can use a networked filesystem (or even a SAN) to distribute it across servers.

via Twitter, Rails, Hammers, and 11,000 Nails per Second | Thought Palace.