Sunday, 18 March 2012

Hourly timestamp in your twitter stream

This is such a silly project that I'm only going to post the code here as a curio. I liked the idea of an hourly time code in my twitter stream, so started following @big_ben_clock which helpfully BONGs on the hour. This is OK if you are in the same time zone as Big Ben and can quickly enumerate the BONGS at, say, 9 O'Clock but useless otherwise.

I thought it would be worth a couple of minutes of my time to convert the tweet text to a localtime time of the tweets generated by @big_ben_clock making the 'service' more universal and to get rid of the BONGS - which wear thin after a while.

So, here's the code for the TTYtter extension: (needs Date::Manip)

#bigben2localtime
use Date::Manip;

# Simply follow @big_ben_clock and this will replace the BONGS with a
# localtime conversion of the tweet timestamp.
# @pr4wn

sub bb_time_convert {
  my $date = shift;
  my $s;

  $s = substr( $date, 8, 5 ); # hh:mm
  return( $s );

}

$handle = sub {


my $ref = shift;

my $ESC = pack("C", 27);
my $BOLDON = ($ansi) ? "${ESC}[1;31m" : '';
my $BOLDOFF = ($ansi) ? "${ESC}[0m" : '';

if ($ref->{user}->{screen_name} eq 'big_ben_clock' ) {
   
  my $created_at = ParseDate($ref->{'created_at'});
  $ref->{text} = $BOLDON . bb_time_convert($created_at) . $BOLDOFF;

}

&defaulthandle ( $ref );

}


Monday, 5 March 2012

Filtering your twitter feed with TTYtter

It's no huge deal, but I have noticed that there are certain categories of tweet that I have no interest in. Having written an extension for TTYtter previously, I thought that I'd knock up an extension to filter out tweets using regexp filters.

Of course, I took the precaution of RTFMing the TTYtter documentation before taking on the task, and discovered that the feature already exists, namely, the filter= directive in the .ttytterrc file. The lesson here is RTFM before re-inventing the wheel.

So, for the time being, I'm experimenting with these filters:

filter=(/Please RT/i|/pl[s|z] RT/i | /[@|#]TheWantApp/i | /#getglue/i | /#nowplaying/i | /[#|@]klout/i)

I'll see how it works out over the next few days and see if it behaves.

Update: The filter above works as advertised which is handy to know. I may consider doing a plugin to extend the capability with lists of rules and being able to use regexps on a wider range of data.

Off the top of my head, a rules file like:

#rule 1
( ( $ref->{user}->{screen_name} eq 'pr4wn' ) && ( $ref->{text} =~ /\blinux\b/i ) )
#rule 2
( $ref->{text} =~ /#dullhashtag/i )
#rule n
:
:

Then go through the rules and omit any matches.

Something for a rainy day, perhaps and not particularly urgent - I'm unsure how much it would be used ;-)

And here we have it. A proof of concept that I ran during Question Time (so I could predict active tweeters and hash tags). Apologies to @DIMBLEBOT ;-)

I've dediced to post code similar to below here. Simply download the ttytterkf tar.gz file and off you go.

#ttytterkf
$handle = sub {

  my $ref = shift; 

  my @filter_rules = ( 
  #Rules can be any legal true/false perl conditon, so fill your boots. 
  '( ( ($ref->{user}->{screen_name} =~ /^pr4wn$/i ) && ($ref->{text} =~ /\blinux\b/i ) )',
   #Some test rules to test during Question Time. 
  '( $ref->{user}->{screen_name} =~ /^DIMBLEBOT$/i )', 
  '( quotemeta($ref->{text} ) =~ /\bdimbledance/i )',
  '( quotemeta($ref->{text} ) =~ /#bbcqt/i )' 
  ); 


  foreach my $rule ( @filter_rules ) { 

    $i = eval $rule; 


    if ( $i == 1 ) { return; } 

  } 


  &defaulthandle( $ref ); 


  return; 


}

Saturday, 10 September 2011

.bashrc settings

In the unlikely event of me losing my custom .bashrc settings, I shall post a copy here as a backup :-)
I tend to use whatever settings come with the distro that I'm using and bung this on the end.

NNTPSERVER='localhost' && export NNTPSERVER
#home server
ORAC='my.home.server.fqdn' && export ORAC

#BAD TYPING KLAXON.
alias suod='sudo'
alias yums='yum search'
alias yumi='sudo yum install'
alias yumsync='sudo yum distro-sync'
alias yumu='sudo yum upgrade'
alias apti='sudo apt-get install'
alias aptu='sudo apt-get upgrade'
alias sshh='gnome-terminal --profile=ssh --title="ssh home" -e "ssh $ORAC"'

alias pacu='sudo pacman -Syu'
alias paci='sudo pacman -S'


#-------------------------------------------------------------
# File functions:
#-------------------------------------------------------------


# Find a file with a pattern in file name
function ff() { find . -type f -iname '*'$*'*' -ls ; }


# Find a file with pattern $1 & Execute $2 on it:
function fe()
{ find . -type f -iname '*'${1:-}'*' -exec ${2:-file} {} \;  ; }


# Find a pattern in a set of files
function fstr()
{
    OPTIND=1
    local case=""
    local usage="fstr: find string in files.
Usage: fstr [-i] \"pattern\" [\"filename pattern\"] "
    while getopts :it opt
    do
        case "$opt" in
        i) case="-i " ;;
        *) echo "$usage"; return;;
        esac
    done
    shift $(( $OPTIND - 1 ))
    if [ "$#" -lt 1 ]; then
        echo "$usage"
        return;
    fi
    find . -type f -name "${2:-*}" -print0 | \
    xargs -0 egrep --color=always -sn ${case} "$1" 2>&- | more 


}
# Useful?  You bet.
function extract()
{
     if [ -f $1 ] ; then
         case $1 in
             *.tar.bz2)   tar xvjf $1     ;;
             *.tar.gz)    tar xvzf $1     ;;
             *.bz2)       bunzip2 $1      ;;
             *.rar)       unrar x $1      ;;
             *.gz)        gunzip $1       ;;
             *.tar)       tar xvf $1      ;;
             *.tbz2)      tar xvjf $1     ;;
             *.tgz)       tar xvzf $1     ;;
             *.zip)       unzip $1        ;;
             *.Z)         uncompress $1   ;;
             *.7z)        7z x $1         ;;
             *)           echo "'$1' cannot be extracted." ;;
         esac
     else
         echo "'$1' is not a valid file"
     fi
}

Tuesday, 9 August 2011

#londonriots

                        pwntterstats


Run on:..........  Thu Aug 11 14:19:47 BST 2011
Stats from:......  08-08-2011
Stats to:........  08-08-2011
Total tweets:....  593
Bytes tweeted:...  57,121

               Most popular hashtags in feed

 1. #londonriots...................................:      104
 2. #enfield.......................................:       12
 3. #vinb..........................................:       11
 4. #tottenham.....................................:        6
 5. #cpfc..........................................:        5
 6. #ubuntu........................................:        4
 7. #debtcrisis....................................:        3
 8. #suckers.......................................:        3
 9. #sigh..........................................:        2
10. #cdwm..........................................:        2
11. #horizon.......................................:        2
12. #motogp........................................:        2
13. #medicalfilms..................................:        2
14. #londonriot....................................:        2
15. #thatsall......................................:        1
16. #leeds.........................................:        1
17. #bbc4..........................................:        1
18. #keepsmilin....................................:        1
19. #crawley.......................................:        1
20. #lameprexercise................................:        1


Hashtag Cloud

#backofthethroat #bbc4 #bcci #brnoorbust #cdwm #cpfc #crawley #debtcrisis #diskriminacija #dontgiveafuggnews #edl #enfield #horizon #justsayin #keepsmilin #klf #lameprexercise #leeds #letthegamesbegin #lhc #londonriot #londonriots #londonsburning #medicalfilms #motogp #ms #multiplesclerosis #murdocracy #newsnight #newspeak #newtwitter #nowshowing #nufc #pbd #peopleofheaven #perspective #physics #randomtweet #raybans #reallysadface #riots #sigh #suckers #thatsall #toomanytweets #tottenham #ubuntu #uk #ukriots #vinb

Generated by pwntterstats version 0.2

Monday, 1 August 2011

pwntterstats

Introducing pwntterstats

pwntterstats is an extension which generates reports from data collected by pwntter.

You will need a working installation of pwntter in order to run this utility.

Download

pwntterstats can be downloaded here.


A sample report is below.


                 pwntterstats Sample Report


Run on:..........  Tue Aug  2 09:06:00 BST 2011
Stats from:......  29-07-2011
Stats to:........  02-08-2011
Total tweets:....  2,176
Bytes tweeted:...  198,112

                 Tweets per Twitter client

 1. TweetDeck......................................:      647
 2. Twitter for iPhone.............................:      525
 3. Tweetlogix.....................................:      458
 4. Twittelator....................................:      103
 5. TTYtter (pr4wn)................................:       66
 6. Twitter for BlackBerry®........................:       66
 7. FBtoTweet......................................:       47
 8. Mobile Web.....................................:       38
 9. Twidroyd for Android...........................:       35
10. Tweet Button...................................:       22
11. Echofon........................................:       22
12. Instagram......................................:       20
13. Seesmic........................................:       16
14. Twitter for iPad...............................:       13
15. Facebook.......................................:       12
16. Tweetbot for iPhone............................:       12
17. Twitter for Mac................................:       12
18. Twitpic........................................:       11
19. Google.........................................:       11
20. HootSuite......................................:        9
21. Twitter for Android............................:        7
22. b2evolution tweet plugin.......................:        4
23. TTYtter........................................:        4
24. twitterfeed....................................:        4
25. bitly..........................................:        3
26. TweetPic for iPhone............................:        3
27. web............................................:        2
28. Choqok.........................................:        2
29. The Guardian iPhone App........................:        1
30. Twitterrific...................................:        1

                   Busiest users in feed

 1. aaaaa..........................................:      287
 2. bbbbbbbb.......................................:      220
 3. cccccccccccccc.................................:      182
 4. ddddddd........................................:      129
 5. eeeeeeee.......................................:      123
 6. fffffffffffffff................................:      108
 7. ggggggggggggg..................................:       90
 8. hhhhhhhhhh.....................................:       70
 9. iiiiiii........................................:       69
10. jjjjjjjjjjj....................................:       63
11. kkkkkkkkkkkkk..................................:       59
12. lllllllllllllll................................:       57
13. mmmmmmmmmm.....................................:       52
14. nnnnnnnnnn.....................................:       52
15. ooooooooooooooo................................:       47
16. ppppppppp......................................:       46
17. qqqqqqqqq......................................:       43
18. rrrrrrrrr......................................:       40
19. sssss..........................................:       38
20. ttttttttttt....................................:       31

               Most popular hashtags in feed

 1. #fridaytwiz....................................:       58
 2. #EngvInd.......................................:       34
 3. #cpfc..........................................:       21
 4. #NUFC..........................................:       14
 5. #celebrityheadchef.............................:       12
 6. #FridayTwiz....................................:       10
 7. #CDWM..........................................:        9
 8. #CDWMIE........................................:        8
 9. #Iran..........................................:        7
10. #F1............................................:        7
11. #bbccricket....................................:        6
12. #newsnight.....................................:        5
13. #ubuntu........................................:        5
14. #ClassicMurdoch................................:        5
15. #Bahrain.......................................:        5
16. #tms...........................................:        5
17. #Papua.........................................:        5
18. #ssj...........................................:        5
19. #weareallanarchists............................:        5
20. #USdebt........................................:        5

                   Busiest hours in feed

 1. 16:00..........................................:      169
 2. 22:00..........................................:      148
 3. 14:00..........................................:      147
 4. 11:00..........................................:      146
 5. 12:00..........................................:      145

                   Quietest hours in feed

 1. 05:00..........................................:        3
 2. 06:00..........................................:        7
 3. 03:00..........................................:        9
 4. 02:00..........................................:       10
 5. 04:00..........................................:       12

                    Busiest days in feed

 1. 2011-07-29.....................................:      625
 2. 2011-07-31.....................................:      521
 3. 2011-07-30.....................................:      508
 4. 2011-08-01.....................................:      458
 5. 2011-08-02.....................................:       64

                   Quietest days in feed

 1. 2011-08-02.....................................:       64
 2. 2011-08-01.....................................:      458
 3. 2011-07-30.....................................:      508
 4. 2011-07-31.....................................:      521
 5. 2011-07-29.....................................:      625

                Posts by day of week in feed

 1. Fri............................................:      625
 2. Sun............................................:      521
 3. Sat............................................:      508
 4. Mon............................................:      458
 5. Tue............................................:       64

Generated by pwntterstats version 0.1

Command Line Options

To demonstrate the flexibility of pwntterstats, here are the command line options:


pwntterstats --help

Usage: $0 [options] 
--usage, --help, -h, -?       Print this message
--from, -f                     From date YYYYMMDD [yesterday]
--to, -t                     To date YYYYMMDD   [yesterday)]
--output, -o Output filename
--title, -e Report title       [pwntterstats]
--width, -w Report Width       [72]
--html, -l                     Simple HTML output
--all, -a Runs all reports except the hashtag cloud 
report unless --html arguent is used
--dow, -w Day of week report
--hashtag, -ht Hashtag report
--client, -c Twitter client report
--user, -u User report
--hours, -ho Busy/quiet hours report
--days, -d Busy/quiet day report
--cloud, -cl Run a hashtag cloud report in html
The rest of the report will be in html too.
--cloudwords Maximum number of words in hashtag cloud [50]
--nocss Do not print CSS code with cloud report


example:
  
pwntterstats --from 20110701 --to 20110731 --all --html --title "July 2011 Statistics" --output /var/www/stats/201107.html 



Wiki

I've created a wiki for any urgent notifications relating to pwntter here.

I draw your attention to this article to reconfigure your time_zone field to allow NULL items to be returned by the twitter API.

Friday, 29 July 2011

Hello Mint

Having got fed up with the direction Canonical are taking with Ubuntu, I have spent the last month flirting with other distros to see which would be a good fit. My only requirement was that it had to have Gnome 2 out of the box and a fairly hassle free install on my laptop.

For the time being, I have gone with Linux Mint Debian Edition which is a heady cocktail of Mint, Debian and Debian-Multimedia repositories and suits my pragmatic approach to non-free extras.  So, it's installed and I have got everything to work – in fact, my desktop looks identical and only the Mint branded GDM login screen gives the game away.

Hardware Issues

Sound


00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA)

This was the first distro not to play nicely with my soundcard and was fixed by downloading hda-verb and appending the following to

/etc/rc.local/usr/local/sbin/hda-verb /dev/snd/hwC0D0 0x01 SET_GPIO_MASK 3
/usr/local/sbin/hda-verb /dev/snd/hwC0D0 0x01 SET_GPIO_DIRECTION 1
/usr/local/sbin/hda-verb /dev/snd/hwC0D0 0x01 SET_GPIO_DATA 1

Don't ask me why, but it just worked.

Broadcom


05:00.0 Network controller: Broadcom Corporation BCM4311 802.11b/g WLAN (rev 01)

As ever, my BCM4311 wifi card failed to be detected and was quickly fixed by:


sudo apt-get install firmware-b43-installersudo modprobe b43

Here's a screenshot.


Friday, 18 March 2011

Setting up TTYtter

I've been using TTYtter on my laptop solidly for 24 hours now and have got it set up just so.

For my perfect recipe, add extension libnotifyperl and install CPAN module Term::ReadLine::TTYtter.

My  .ttytterrc file looks like this:
ansi=1
readline=Term::ReadLine::TTYtter
vcheck=1
slowpost=4
exts=/usr/local/bin/libnotifyperl.pl
notifytype=libnotifyperl
notifies=dm,reply
#sod ASCII art, I opted to call eye of gnome to view avatars ;-)
avatar=curl -s %U > /tmp/ttytter-%N.%E; eog /tmp/ttytter-%N.%E &