Both mother and baby are doing well...
The often deranged postings of yet another hacker, pretending to be an Astronomer, pretending to be a hacker who has written a book or two for O'Reilly Media.
Thursday, July 31, 2008
Interrupted Journeys
For those of you puzzled by my early departure from OSCON a week ago, and my non-appearance at HTN IV this week, I'd like to announce the very unexpected early arrival of my son, Alexander Michael. Born late yesterday evening, just under two months premature, and weighing just over 4 lbs.
Both mother and baby are doing well...
Both mother and baby are doing well...
Wednesday, July 23, 2008
OSCON: Wednesday Morning Keynote
My jet lag caught up with me last night and I ended not making it to the Tuesday Night Extravaganza, although other people did and cruelly didn't blog Damian's talk for the rest of us that didn't. So I don't get to find anything more about "Temporally Quaquaversal Virtual Nanomachine Programming In Multiple Topologically Connected Quantum-Relativistic Parallel Timespaces", which is a pity...
The keynote kicked off with Allison Randall and Edd Dumbill, talking about the history of Open Source and OSCON. This is the first OSCON without Nat at the helm, and while I've seen him around, it's pretty weird not to have the keynote kick off with "...and here's your conference chair, Nat Torkington".
Update: Looks like I'll see you all next year. While I was in the keynote I got a phone call and I'm now heading back into the UK somewhat earlier than planned.
Update: More than one interrupted journey...
The keynote kicked off with Allison Randall and Edd Dumbill, talking about the history of Open Source and OSCON. This is the first OSCON without Nat at the helm, and while I've seen him around, it's pretty weird not to have the keynote kick off with "...and here's your conference chair, Nat Torkington".
Update: Looks like I'll see you all next year. While I was in the keynote I got a phone call and I'm now heading back into the UK somewhat earlier than planned.
Update: More than one interrupted journey...
Perl on Google App Engine
I woke up this morning to some of the best news I've heard in a while, it looks like there is some progress with putting Perl onto Google App Engine.
More from Brad Fitzpatrick. If you'd like to discuss this or help out, join the perl-appengine mailing list, and submit code to the appengine-perl project on Google Code. For more information see the Perl-on-AppEngine FAQ.
Maybe I won't have to learn Python after all...
More from Brad Fitzpatrick. If you'd like to discuss this or help out, join the perl-appengine mailing list, and submit code to the appengine-perl project on Google Code. For more information see the Perl-on-AppEngine FAQ.
Maybe I won't have to learn Python after all...
Labels:
Cloud Computing,
Google,
Google App Engine,
Perl,
Web 2.0,
Web Services
Tuesday, July 22, 2008
OSCON: Practical Erlang Programming
This afternoon I'm sitting in "Practical Erlang Programming" given by Francesco Cesarini. Erlang has been around for almost twenty years, and is a niche language. However we're increasingly starting to hear more about it due to the growth in the number of multi-core machines. So I figured I so go and find out what all the fuss was about...
Update: I think Francesco has really overestimated the capacity of the wireless network, he's just told ninety people to download the source bundle and install Erlang.
Update: Okay, we're kicking off with data types; integers, floats, atoms are the simple types. Then we have tuples and lists. Interestingly variables in Erlang are single assignment, an values of variables can not be changed once it has been bound. Puzzled, variables are not very variable at that point?
Update: Pattern matching is used for assigning vales to variables, controlling the executing flow of programs and extracting values.
Update: Moving onto to function calls, this looks, well. Odd.
Function have clauses separated by '
Variables are local to functions and allocated and deallocated automatically.
Update: Modules are stored in files with the
compiling this from the command line
Update: We've now looked at the basics, we're moving on to sequential Erlang; Conditionals, guards and recursion.
In a conditional one branch must always succeed, you can put the '
Again, one branch must always succeed, by using
instead of having this,
Of these two the top one is the faster one, but you really shouldn't really worry about that when using Erlang, apparently...
All variables in guards have to be bound. If all guards have to succeed, use '
Update: On to recursion,
Note the pattern of recursion is the same in both cases. Taking a list and evaluating an element is a very common pattern...
Update: Now onto Build In Functions (BIFs)...
BIFs are by convention regarded as being in the erlang module. There are BIFs for process and port handling, object access and examination, meta programming, type conversion, etc.
Update: We're running through all the possible run time errors.
Update: We're breaking (late!) for coffee...
Update: ...and we're back, and walking through some examples, and onwards to concurrent Erlang.
Before the spawn code is executed by Pid1, afterwards a new process Pid2 is created. The identified Pid2 is only known to Pid1. A process terminates abnormally when run-time error occurs, and normally when there is no more code to execute. Processes do not share data, and the only way to do so is using message passing. Sending a message will never fail, messages sent to non existing processes are thrown away. Received messages are stored in a process mailbox, and will receive them inside a receive clause,
Unlike a
Update: We're getting into a static versus dynamic typing argument, the bizarreness is that even the Francesco seems to think that static typing is a good thing. Why is that? I'm really surprised, after all I'd argue that there are a bunch of reasons to use loosely typed languages in preference to statically typed ones.
Update: It's also interesting that some people in the audience here aren't getting the "let it crash" mantra coming from Francesco. In a highly concurrent language where everything is a process, letting a process crash is just how you handle errors. A process crash is essentially the same as throwing an exception.
Update: I'm starting to loose the thread of the talk now. Pity, Francesco has just got to the interesting bit. It's been a long day...
Update: ...and we're done. Chris was also blogging the tutorial so head over to his post for more coverage.
Update: I think Francesco has really overestimated the capacity of the wireless network, he's just told ninety people to download the source bundle and install Erlang.
Update: Okay, we're kicking off with data types; integers, floats, atoms are the simple types. Then we have tuples and lists. Interestingly variables in Erlang are single assignment, an values of variables can not be changed once it has been bound. Puzzled, variables are not very variable at that point?
1> A = 123.
123
2> A.
123
3> A = 124.
** exception error no match of right hand
4> f().
ok
5> A = 124.
124
Update: Pattern matching is used for assigning vales to variables, controlling the executing flow of programs and extracting values.
Update: Moving onto to function calls, this looks, well. Odd.
area( {square, Side} ) ->
Side * Side ;
area( {circle, Radius } ) ->
3.14 * Radius * Radius;
area( {triangle, A, B, C} ) ->
S = ( A + B + C )/2,
math:sqrt(S*(S-A)*(S-B)*(S-C));
area( Other ) ->
{error, invalid_object}.
Function have clauses separated by '
;
'. Erlang programs consist of a collection of modules that contain functions that call each other. Function and modules names must be atoms.factorial(0) ->
1;
factorial(N) ->
N * factorial(N-1).
Variables are local to functions and allocated and deallocated automatically.
Update: Modules are stored in files with the
.erl
suffix, module and file names must be the same. Modules are names with the -module(Name).
directive.-module(demo).
-export([double/1]).
% Exporting the function double with arity 1
double(X) ->
times(X, 2).
times( X, N ) ->
X * N.
compiling this from the command line
1> cd("/Users/aa/").
2> c(demo).
{ok,demo}
3> demo:double(10).
20
4> demo:times(1,2)
**exception error: undefined function demo:times
Update: We've now looked at the basics, we're moving on to sequential Erlang; Conditionals, guards and recursion.
case lists:members(foo, List) of
true -> ok;
false -> {error, unknown}
end
In a conditional one branch must always succeed, you can put the '
_
' or an unbound variable in the last clause to ensure this happens.if
X < 1 -> smaller;
X > 1 -> greater ;
X == 1 -> equal
end
Again, one branch must always succeed, by using
true
as the last guard you ensure that the last clause will always succeed should previous ones evaluate to false, see it as an 'else' clause. So we can have,factorial(N) when N > 0 ->
N * factorial( N - 1 );
factorial(0) ->
1.
instead of having this,
factorial(0) ->
1;
factorial(N) ->
N * factorial(N-1).
Of these two the top one is the faster one, but you really shouldn't really worry about that when using Erlang, apparently...
All variables in guards have to be bound. If all guards have to succeed, use '
,
' to seperate them, if one has to succeed, use ';
' to separate them. Guards have to be free of side effects.Update: On to recursion,
average(X) -> sum(X) / len (X).
sum([H|T) -> H + sum(T);
sum([]) -> 0.
len([_|T]) -> 1 + len(T);
len([]) -> 0.
Note the pattern of recursion is the same in both cases. Taking a list and evaluating an element is a very common pattern...
Update: Now onto Build In Functions (BIFs)...
date()
time()
length(List)
size(Tuple)
atom_to_list(Atom)
list_to_tuple(List)
integer_to_list(234)
tuple_to_list(Tuple)
BIFs are by convention regarded as being in the erlang module. There are BIFs for process and port handling, object access and examination, meta programming, type conversion, etc.
Update: We're running through all the possible run time errors.
Update: We're breaking (late!) for coffee...
Update: ...and we're back, and walking through some examples, and onwards to concurrent Erlang.
Pid2 = spawn(Mod, Func, Args)
Before the spawn code is executed by Pid1, afterwards a new process Pid2 is created. The identified Pid2 is only known to Pid1. A process terminates abnormally when run-time error occurs, and normally when there is no more code to execute. Processes do not share data, and the only way to do so is using message passing. Sending a message will never fail, messages sent to non existing processes are thrown away. Received messages are stored in a process mailbox, and will receive them inside a receive clause,
recieve
{resetn, Board } -> reset(Board);
{shut_down, Board{ -> {error, unknown_msg}
end
Unlike a
case
block, receive suspends the process until a message which matches a case is received. Message passing is asynchronous, one of the things you look for in stress testing Erlang systems is running out of memory because of full mailboxes.Update: We're getting into a static versus dynamic typing argument, the bizarreness is that even the Francesco seems to think that static typing is a good thing. Why is that? I'm really surprised, after all I'd argue that there are a bunch of reasons to use loosely typed languages in preference to statically typed ones.
Update: It's also interesting that some people in the audience here aren't getting the "let it crash" mantra coming from Francesco. In a highly concurrent language where everything is a process, letting a process crash is just how you handle errors. A process crash is essentially the same as throwing an exception.
Update: I'm starting to loose the thread of the talk now. Pity, Francesco has just got to the interesting bit. It's been a long day...
Update: ...and we're done. Chris was also blogging the tutorial so head over to his post for more coverage.
Labels:
Erlang,
Francesco Cesarini,
OSCON,
OSCON08,
OSCON2008
OSCON: Perl Worst Practices
So after coffee I've bailed from "An Open Source Startup in Three Hours" into "Perl Worst Practices" with Damian Conway, squeezing myself into one of the few empty seats.
Damian is going through his SelfGOL code, and oh boy has he done some evil things with Perl...
Just like Chris I'm looking forward to the prospect of putting these worst practices into my own code when I return to my work next week.
Brad has been sitting in this one since the beginning, so if you're really interested in what Damian has been saying then just jump over to his blog post for the principles of worst practice...
Damian is going through his SelfGOL code, and oh boy has he done some evil things with Perl...
You have to minimise wear and tear on all parts of the keyboard
Just like Chris I'm looking forward to the prospect of putting these worst practices into my own code when I return to my work next week.
Far too few people are using their source code as their user interface
Brad has been sitting in this one since the beginning, so if you're really interested in what Damian has been saying then just jump over to his blog post for the principles of worst practice...
OSCON: An Open Source Startup in Three Hours
The second day of OSCON, and this morning I'm sitting in "An Open Source Startup in Three Hours" given by Gavin Doughtie and Andrew Hyde.
It's an odd choice for an academic like me, but considering the current funding crisis perhaps not as inexplicable as all that...
Update: The trick to running a successful company is to work half a day, work the first twelve hours or the second twelve hours it doesn't matter. It's not about getting rich quick, or about pyramids. Andrew is talking about his startup weekends...
He's talking about deliverables and paper prototyping. Obviously they're not going to code it up, launch a project and get acquired by lunch time, but hopefully they're going to give us the tools to think about how to do that.
Update: Paper prototyping allows you to see things quickly, and avoid those six month mistakes and to fail really fast. Startups succeed with the right team and a solid idea. But it comes down to timing and luck and good design. But good designers are hard to find.
Update: A startup has to solve problems, but it has to solve more than just your problem. You need to float your idea to as many people as possible, and as smart as people as you can find. Unlike a lot of people Andrew is arguing that if people tell you your going to fail, either your pitching your idea wrong, or you're working on the wrong thing.
Update: What not to do; think your idea is worth anything, build anything that is "neat" or "cute", define development or design early on, or pick a domain. If you pick up a domain before you talk to people you probably haven't developed your idea well enough. You have to define your goals, build stuff, launch it quickly and see what people think.
Update: Define what your idea is and isn't, define the problem and outline your solution to it. Andrew is really hitting on paper prototyping you idea, design and interface. I'm pretty surprised, while I've never heard it called that before this is something I always do for a project, he's also hitting on what he's calling 'moleskinning' which seems to be the equivalent of the good old laboratory note book. You write down ideas, steps you've taken, that sort of thing. Maybe a rigorous science background is a good thing for a startup?
Update: Design is important, and a lot of people get it wrong. Go to someplace like CSS Beauty and find out what's hot and what's not.
Update: Over to Gavin and implementation. Things you need before launch; source control, a sever, some programming and you need to deploy something. You might want to think about a 'light engineering' process. The lightest engineering Gavin can think of is '1-click installs' and content based startups.
Update: He's now talking about Amazon EC2. If anyone is sitting in a startup tutorial and doesn't know about EC2 (and S3) then they're in trouble. He's walking us through building an EC2 image though, which is quite interesting. I've played with S3, but not EC2 in any depth. He's pointing out that you when you're doing development you don't have to keep the instances running, Amazon will charge you for those running instances, just shut them down. However when you do shut them down, or they crash, any data in the instance is gone, so you have to push it off to long term storage like S3.
Update: Lighter weight than EC2 is Google App Engine, which (as we all know?) is a sandbox Python runtime... and now we're getting a demo. App Engine is actually something I've started to play with as I managed to pick up an invite...
Update: He's pointed us towards opendesigns.org for (free) design templates, and encouraging us to put real design into your startup as soon as possible because it'll help you think about it more seriously.
Update: Gavin's talking about Django,
Update: After fiddling around with CSS templates and Dojo, which looks pretty cool by the way, Gavin is back deploying his prototype App Engine project and showing how the group development stuff works...
Update: Back to talking about startups. What you don't need is; scalability, performance or industrial strength anything, unless of course that's what your startup is about. Technological pitfalls; Java, no local expertise and being overambitious. No really, he's telling us not to use Java. There is lots of complexity in Java land, and he really would recommend looking at the LAMP stacks.
Update: Even though it's really obvious stuff a simple application may have some technical depth to it. So don't be overambitious...
Update: Remember your team is your technology. Don't be afraid to have a single point of failure if that gives you leverage. Spend a bit of time looking for leverage in your technology.
Update: Marketing is a big thing. If you build it, they won't necessarily come...
Update: Andrew is back and talking about feature creep, which is the number two killer of startups, right after running out of money. You need a kerbside pitch, and an elevator pitch. A kerbside pitch is real fast, essentially "I work at etc company, which is like foo except for bar". Every one on the team should be able to do the elevator pitch, the longer (slightly more involved) pitch.
Update: Back to tools, pointing us at JumpBox which isn't something I'd run across before...
Update: Over to talking about legal stuff. You need to get everyone that you've ever talked to about the project, that has ever given any input to it, to sign bits of paper and state their intent.
...and software patents are evil if you don't have them!
Update: We're breaking for coffee, I think I'm going to jump out of this tutorial and try and slip into the second half of Damian's Perl Worst Practices after coffee. This stuff is interesting, but they're really talking about startups as websites, which while okay wasn't really what I was here for...
Update: Jumped over into Damian's talk.
It's an odd choice for an academic like me, but considering the current funding crisis perhaps not as inexplicable as all that...
Update: The trick to running a successful company is to work half a day, work the first twelve hours or the second twelve hours it doesn't matter. It's not about getting rich quick, or about pyramids. Andrew is talking about his startup weekends...
Our company was built in 3 hours, the rest was marketing bullshit - Bill Gates
Amazing product are never built in a long time. Just short, simple acid trips - Steve Jobs
He's talking about deliverables and paper prototyping. Obviously they're not going to code it up, launch a project and get acquired by lunch time, but hopefully they're going to give us the tools to think about how to do that.
Update: Paper prototyping allows you to see things quickly, and avoid those six month mistakes and to fail really fast. Startups succeed with the right team and a solid idea. But it comes down to timing and luck and good design. But good designers are hard to find.
Update: A startup has to solve problems, but it has to solve more than just your problem. You need to float your idea to as many people as possible, and as smart as people as you can find. Unlike a lot of people Andrew is arguing that if people tell you your going to fail, either your pitching your idea wrong, or you're working on the wrong thing.
Update: What not to do; think your idea is worth anything, build anything that is "neat" or "cute", define development or design early on, or pick a domain. If you pick up a domain before you talk to people you probably haven't developed your idea well enough. You have to define your goals, build stuff, launch it quickly and see what people think.
Update: Define what your idea is and isn't, define the problem and outline your solution to it. Andrew is really hitting on paper prototyping you idea, design and interface. I'm pretty surprised, while I've never heard it called that before this is something I always do for a project, he's also hitting on what he's calling 'moleskinning' which seems to be the equivalent of the good old laboratory note book. You write down ideas, steps you've taken, that sort of thing. Maybe a rigorous science background is a good thing for a startup?
Update: Design is important, and a lot of people get it wrong. Go to someplace like CSS Beauty and find out what's hot and what's not.
Update: Over to Gavin and implementation. Things you need before launch; source control, a sever, some programming and you need to deploy something. You might want to think about a 'light engineering' process. The lightest engineering Gavin can think of is '1-click installs' and content based startups.
Update: He's now talking about Amazon EC2. If anyone is sitting in a startup tutorial and doesn't know about EC2 (and S3) then they're in trouble. He's walking us through building an EC2 image though, which is quite interesting. I've played with S3, but not EC2 in any depth. He's pointing out that you when you're doing development you don't have to keep the instances running, Amazon will charge you for those running instances, just shut them down. However when you do shut them down, or they crash, any data in the instance is gone, so you have to push it off to long term storage like S3.
Update: Lighter weight than EC2 is Google App Engine, which (as we all know?) is a sandbox Python runtime... and now we're getting a demo. App Engine is actually something I've started to play with as I managed to pick up an invite...
Update: He's pointed us towards opendesigns.org for (free) design templates, and encouraging us to put real design into your startup as soon as possible because it'll help you think about it more seriously.
Update: Gavin's talking about Django,
The Django guys will say 'Don't say it's like rails,' but it is like rails.Which reminds me, I must take a serious look at Catalyst at some point...
Update: After fiddling around with CSS templates and Dojo, which looks pretty cool by the way, Gavin is back deploying his prototype App Engine project and showing how the group development stuff works...
Update: Back to talking about startups. What you don't need is; scalability, performance or industrial strength anything, unless of course that's what your startup is about. Technological pitfalls; Java, no local expertise and being overambitious. No really, he's telling us not to use Java. There is lots of complexity in Java land, and he really would recommend looking at the LAMP stacks.
Update: Even though it's really obvious stuff a simple application may have some technical depth to it. So don't be overambitious...
Update: Remember your team is your technology. Don't be afraid to have a single point of failure if that gives you leverage. Spend a bit of time looking for leverage in your technology.
Update: Marketing is a big thing. If you build it, they won't necessarily come...
Update: Andrew is back and talking about feature creep, which is the number two killer of startups, right after running out of money. You need a kerbside pitch, and an elevator pitch. A kerbside pitch is real fast, essentially "I work at etc company, which is like foo except for bar". Every one on the team should be able to do the elevator pitch, the longer (slightly more involved) pitch.
Update: Back to tools, pointing us at JumpBox which isn't something I'd run across before...
Update: Over to talking about legal stuff. You need to get everyone that you've ever talked to about the project, that has ever given any input to it, to sign bits of paper and state their intent.
If you can't make it good, at least make it look good - Bill Gates
Intellectual property has the shelf life of a banana - Bill Gates
...and software patents are evil if you don't have them!
Update: We're breaking for coffee, I think I'm going to jump out of this tutorial and try and slip into the second half of Damian's Perl Worst Practices after coffee. This stuff is interesting, but they're really talking about startups as websites, which while okay wasn't really what I was here for...
Update: Jumped over into Damian's talk.
Labels:
Andrew Hyde,
Gavin Doughtie,
OSCON,
OSCON08,
OSCON2008,
Portland,
Startup
Monday, July 21, 2008
OSCON: Perl Security
This afternoon I'm breaking my no Perl rule and I'm sitting in "Perl Security" given by Paul Fenwick. I'm sitting next to Brad and Chris, and Sam is supposedly in here somewhere...
From the looks of things, Paul is trying to pull a Dick Hardt with his talk. If you've ever seen Dick's Identity 2.0 talk you'll know that this tutorial is going to be virtually un-bloggable...
Update: First pointer is to his own
Update: Paul is talking about exposing vulnerabilities in CGI code, and trying to emphasize that you must validate your input data, from users, from files and very much from the network. You need to use taint mode. He's also recommending that you don't use "baby taint" mode and call Perl with
Update: Interesting he's just shown a three line example that will allow you to get root access by running any Perl script that respects the
Update: But there are problems with taint mode
Update: One of the things that Perl is good for is making tasks easy. One of those things is opening a file. There are lots of ways to open files in Perl, which means there are lots of way to break things. If you are opening a file you should always specify a mode, and never use the two argument version of
Update: Symbolic links (are awesome but) can be used to do evil things. Don't use temporary files with predictable names, you could even use anonymous files
or you could just use the Tim's
Update: Now talking about
if
which is rather bad.
Back ticks are worse; "this" is a string, 'this' is a string and `this` is executing arbitrary code on your system. Depending on your font, it's rather hard to tell the difference.
The alternative is
Update: We're breaking for afternoon coffee...
Update: ...and we're back and talking about
Update: On to database security and injection attacks. He's advising us not to do our own quoting and use instead use place holders. As I've mentioned before, I rarely have to do heavy weight database work, and I'd view my knowledge of DBI as only fairly sparse, and even I know about place holders. Why does everyone think they're worth talking about at length. Bemusing!
Update: On to tricks and tips... and the poisoned null byte. Strings in Perl (are awesome but) can contain any character you want, including control codes and null bytes. In Perl a null byte is just another character, but in C it represents the end of a string. So if you can get a null byte down to the C layer, bad things can happen, and it's easy to pass a null byte can be easily passed in URLs.
Update: ...and we're done (with the main material) about an hour ahead of time. The course notes, although not the slides, for the tutorial are available online. On to the 'bonus' material, random numbers and cryptography...
Update: ...we're done. Now go read Brad's and Chris' posts on the Perl Security tutorial as well.
From the looks of things, Paul is trying to pull a Dick Hardt with his talk. If you've ever seen Dick's Identity 2.0 talk you'll know that this tutorial is going to be virtually un-bloggable...
Update: First pointer is to his own
autodie
module, which was released onto CPAN (very) late last night. It's a lexical equivalent of Fatal
for Perl 5.10, it also upgrades the Fatal
module to contain better diagnostics and reporting.Update: Paul is talking about exposing vulnerabilities in CGI code, and trying to emphasize that you must validate your input data, from users, from files and very much from the network. You need to use taint mode. He's also recommending that you don't use "baby taint" mode and call Perl with
-t
...You may not use data derived from outside your program to affect something else outside your program - at least, not by accident. - perlsec
Update: Interesting he's just shown a three line example that will allow you to get root access by running any Perl script that respects the
PERL5LIB
environment variable.Update: But there are problems with taint mode
If the regular expression match formy ($url, $file) = @ARGV;
$url =~ m/ pattern match /;
my $safe_url - $1;
$file =~ / pattern match /;
my $safe_file = $1;
$safe_file
fails then $safe_file
is set to $safe_url
and that could easily be set to something like http://foo&/bin/sh
, which would be bad. You always need to check for success, either use an if( )
block or by explicitly check the return value from the regular expression.Update: One of the things that Perl is good for is making tasks easy. One of those things is opening a file. There are lots of ways to open files in Perl, which means there are lots of way to break things. If you are opening a file you should always specify a mode, and never use the two argument version of
open
which does much deep magic. Use the three argument version instead.However this doesn't get you out of validating your input, you still need to do that...open($fh, "<", $filename );
Update: Symbolic links (are awesome but) can be used to do evil things. Don't use temporary files with predictable names, you could even use anonymous files
use autodie qw(open);
open($fh, "+>", undef );
or you could just use the Tim's
File::Temp
module instead...Update: Now talking about
system
and back ticks, Paul really hates system
. You really need to use the two argument version of system
rather than the one argument version, except of course@args = ();
system( $cmd, @args );
if
@args
is empty, then it calls the one argument version of system
, which in turn calls the shell rather than executes the passed command directly. Which means that you can this happens,system ( "finger $username; rm -rf *", ( ) );
which is rather bad.
Back ticks are worse; "this" is a string, 'this' is a string and `this` is executing arbitrary code on your system. Depending on your font, it's rather hard to tell the difference.
The alternative is
IPC::System::Simple
, which works all the way back to Pelr 5.6 and (magically) doesn't have any dependencies. Which from the sounds of it goes a long way to fixing a lot of the problems with system
. It also provides capture
which replaces back ticks.Update: We're breaking for afternoon coffee...
Update: ...and we're back and talking about
setuid
and setgid
programs. Apparently Perl is ignorant of the saved uid, which means that it's really hard to drop privileges and make it stick. Also a dark secret, Perl's $<
and $>
variables are cached. Ouch!Update: On to database security and injection attacks. He's advising us not to do our own quoting and use instead use place holders. As I've mentioned before, I rarely have to do heavy weight database work, and I'd view my knowledge of DBI as only fairly sparse, and even I know about place holders. Why does everyone think they're worth talking about at length. Bemusing!
Update: On to tricks and tips... and the poisoned null byte. Strings in Perl (are awesome but) can contain any character you want, including control codes and null bytes. In Perl a null byte is just another character, but in C it represents the end of a string. So if you can get a null byte down to the C layer, bad things can happen, and it's easy to pass a null byte can be easily passed in URLs.
Update: ...and we're done (with the main material) about an hour ahead of time. The course notes, although not the slides, for the tutorial are available online. On to the 'bonus' material, random numbers and cryptography...
Update: ...we're done. Now go read Brad's and Chris' posts on the Perl Security tutorial as well.
OSCON: Python in 3 Hours
While I've written stuff in Java, Python and half a dozen other languages, I'm a Perl guy. My first solution to a programming problem is to use Perl. This hasn't always been the case of course, I used to be a Fortran guy after all, and my solution to most things would be to crank out some Fortran. I'm more or less language agnostic, and try and pick the right tool for every job, it's that so many jobs can be done easily in Perl.
However I've learned Python several times now, used it for whatever project I've needed it for, and then mostly forgotten most of what I knew about it afterwards. But with the release of Google's App Engine however, I think it's time to dust my Python off and pick it up again for a third time. Which is why I'm sitting in "Python in 3 Hours" given by Steve Holden. Steve has put his slides online, but it looks like his hosting provider isn't holding up that well...
Update: The main principle in Python is "Keep it simple stupid", as opposed to Perl's "There is more than one way to do things". That basically sums up the difference between the two languages. It looks like Twitter is over capacity again, the fail whale is very much in evidence.
Update: One of the things that people coming from the C world find difficult is that while Python is a strongly typed language the type is associated with the value, not with the variable. Well that and the mandatory indenting.
Steve is really emphasising the presence of the interactive interpreter, which is something that Python talk about a lot. As a Perl person, it's not something we really have, and I've not felt that lack. Trying to get my head around why Python people feel it's important.
Update: Steve is talking about the different implementations of Python. Also not something we've had with Perl, at least until Perl 6 came along...
Update: That's interesting. Since Python 2.5 integers are unbounded, and since Python 3.0 diving two integers no longer does integer division, you end up with a floating point number. Oh, and strings are going to become Unicode by default.
Update: Now here is something I've always wanted in Perl, and it doesn't have because of its admittedly quite odd object model, default methods on strings;
Having things like strings and tuples, and dictionary keys, as immutable is one of the things I've always had problems with in Python, and yes I know it means it's all thread-safe. Guess I've been working with Perl's threading model too long as well...
Update: Interesting, if obvious, admission from Steve,
Update: The
Update: Time for morning coffee, back after these short messages...
Update: ...and we're back, and Steve has moved on to defining functions, where Python is actually a lot different than most other languages. In Python
Update: After working with Perl for so long I've never liked the way Python handles variable parameter lists,
Update: Pointer to PEP8, the "Style Guide to Python". I guess Perl people just have "Perl Best Practices" by Damian?
Update: Must admit I'm starting to drift here...
I'm trying to figure out whether Python is more strict and complicated than Perl, or whether I'm just thinking in Perl, and Python is just as slack and easy to use as Perl but in a different way? Anyone got any opinions on that one?
Update: I didn't realise the Python interpreter implicitly provides a
Update: Steve just said,
Update: ...and we're done, or at least I am. Time for lunch.
However I've learned Python several times now, used it for whatever project I've needed it for, and then mostly forgotten most of what I knew about it afterwards. But with the release of Google's App Engine however, I think it's time to dust my Python off and pick it up again for a third time. Which is why I'm sitting in "Python in 3 Hours" given by Steve Holden. Steve has put his slides online, but it looks like his hosting provider isn't holding up that well...
Update: The main principle in Python is "Keep it simple stupid", as opposed to Perl's "There is more than one way to do things". That basically sums up the difference between the two languages. It looks like Twitter is over capacity again, the fail whale is very much in evidence.
Update: One of the things that people coming from the C world find difficult is that while Python is a strongly typed language the type is associated with the value, not with the variable. Well that and the mandatory indenting.
Steve is really emphasising the presence of the interactive interpreter, which is something that Python talk about a lot. As a Perl person, it's not something we really have, and I've not felt that lack. Trying to get my head around why Python people feel it's important.
Update: Steve is talking about the different implementations of Python. Also not something we've had with Perl, at least until Perl 6 came along...
Update: That's interesting. Since Python 2.5 integers are unbounded, and since Python 3.0 diving two integers no longer does integer division, you end up with a floating point number. Oh, and strings are going to become Unicode by default.
Update: Now here is something I've always wanted in Perl, and it doesn't have because of its admittedly quite odd object model, default methods on strings;
s.lower()
, s.upper()
, s.strip()
and the like. Of course since strings are immutable, they don't do an in place operation.Having things like strings and tuples, and dictionary keys, as immutable is one of the things I've always had problems with in Python, and yes I know it means it's all thread-safe. Guess I've been working with Perl's threading model too long as well...
Update: Interesting, if obvious, admission from Steve,
We're talking about a Turing complete language, so there is always more than one way of doing things...when talking about there being only one way of doing things in Python.
Update: The
print
statement is actually going away in Python 3.0, replaced by the print
function. Which, yes, makes a big difference. That's a subtle but fairly fundamental change. Have to think about that for a bit...Update: Time for morning coffee, back after these short messages...
Update: ...and we're back, and Steve has moved on to defining functions, where Python is actually a lot different than most other languages. In Python
def
is an executable statement. The interpreter reads and compiles the indented body and the resulting function object is bound to the function name in current namespace.Update: After working with Perl for so long I've never liked the way Python handles variable parameter lists,
Some functions can take an indefinite number of positional and keyword arguments. These are caught with a special parameter syntax;which is only sort of like the way Perl does things...*name
becomes a tuple of "extra" positional arguments to a call**name
becomes a dict of "extra" keyword arguments to a call:n=v
becomes{n: v}
Update: Pointer to PEP8, the "Style Guide to Python". I guess Perl people just have "Perl Best Practices" by Damian?
Update: Must admit I'm starting to drift here...
I'm trying to figure out whether Python is more strict and complicated than Perl, or whether I'm just thinking in Perl, and Python is just as slack and easy to use as Perl but in a different way? Anyone got any opinions on that one?
Update: I didn't realise the Python interpreter implicitly provides a
$self
when using an instance. That's interesting...Update: Steve just said,
Five minutes left to cover formatting and I/O...That's not good. Nor is the fact that it's only in "modern" Python that you can iterate over a file and don't have to read it in all at once. Hmm...
Update: ...and we're done, or at least I am. Time for lunch.
OSCON 2008
Day one of OSCON 2008 here in Portland, Oregon. In my current jet lagged state it was actually fairly easy to haul myself out of bed early enough to make it down to the convention centre and grab morning coffee before everything kicks off for the day. For my body at least its just coming up to four o'clock in the afternoon...
View Larger Map
This year I've decided to take my own advice and keep clear of the Perl track, at least for the tutorials. So this morning I'm going to "Python in Three Hours", then this afternoon I'm going to, erm, "Perl Security". Which is about as far as I could pull myself away from the Perl track for today. Tomorrow I'm off to "An Open Source Startup in Three Hours" and "Practical Erlang Programming".
As always, I'll be blogging things, taking pictures and this year probably twittering the odd comment or two. That's if the fail whale isn't sighted...
Update: Steve Holden talking about Python in Three Hours.
Update: Paul Fenwick talking about Perl Security.
Update: Gavin Doughtie and Andrew Hyde talking about An Open Source Startup in Three Hours.
Update: Bailed out of An Open Source Startup in Three Hours after coffee and into Damian Conway's talk on Perl Worst Practices.
Update: Francesco Cesarini talking about Practical Erlang Programming.
Update: With the tutorials over, the conference is kicking off properly with the Wednesday Morning Keynote.
Update: Looks like I'll see you all next year. While I was in the keynote I got a phone call and I'm now heading back into the UK somewhat earlier than planned.
Update: More than one interrupted journey...
View Larger Map
This year I've decided to take my own advice and keep clear of the Perl track, at least for the tutorials. So this morning I'm going to "Python in Three Hours", then this afternoon I'm going to, erm, "Perl Security". Which is about as far as I could pull myself away from the Perl track for today. Tomorrow I'm off to "An Open Source Startup in Three Hours" and "Practical Erlang Programming".
As always, I'll be blogging things, taking pictures and this year probably twittering the odd comment or two. That's if the fail whale isn't sighted...
Update: Steve Holden talking about Python in Three Hours.
Update: Paul Fenwick talking about Perl Security.
Update: Gavin Doughtie and Andrew Hyde talking about An Open Source Startup in Three Hours.
Update: Bailed out of An Open Source Startup in Three Hours after coffee and into Damian Conway's talk on Perl Worst Practices.
Update: Francesco Cesarini talking about Practical Erlang Programming.
Update: With the tutorials over, the conference is kicking off properly with the Wednesday Morning Keynote.
Update: Looks like I'll see you all next year. While I was in the keynote I got a phone call and I'm now heading back into the UK somewhat earlier than planned.
Update: More than one interrupted journey...
Friday, July 18, 2008
Enroute to OSCON
I'm currently holed up in the Renaissance London Heathrow on my way out to OSCON 2008 being held in Portland in Oregon, at the Oregon Convention Centre.
Unlike last year the journey was fairly pleasant, just over two and a half hours in a straight run down the east coast main line from Leeds, where I was watching my wife graduate with her doctorate, instead of a torturous seven and a half hour journey across a flood torn west country.
However I've got a four thirty am check-in tomorrow morning, so right now it's time to get some sleep. More from me when I hit Portland...
Update: We took off late from Heathrow due to the late arrival of the equipment, we lost more time in the air due to adverse headwinds, and yet more time racked and stacked over a fog-bound O'Hare. So while I'm now on the ground in Chicago, I'm here two hours later than I should have been. Fortunately, for various values of fortunately, my trip through US immigration was relatively unscathed, and my onward flight turns out to be just as delayed going out as I was coming in. So I'm now camped out in the newly rennovated Red Carpet Club near B18. I'll let you know how things go when, or if, I make it into Portland later today...
Update: Finally made it into Portland. My connecting flight out of O'Hare was delayed, delayed again, and then delayed again. Then suddenly moved forward by an hour without warning, to a slot only a few minutes away. Apparently our plane was still sitting in Toronto, and United had decided to give us a different one at short notice. Presumably some other people were now without a plane, however...
This meant that I had to run across the terminal. Although I made it onto the flight ahead of most people, who no doubt had retired to the bar, or just taken a nap secure in the knowledge that had at least another hour to wait.
I also made it onto the plane before the flight crew, who took another hour and a half to arrive after we'd fully boarded. The cabin crew were looking distinctly nervous towards the end...
Once the flight crew arrived, did the safety check and the pre-fight, we'd closed the doors and started to pushback, one of the cabin crew noticed we had an extra passenger. The plane was full, every seat was taken, but there was still one poor guy still standing in the aisle clutching a valid boarding pass, but without a seat. After a brief discussion about whether he could sit in a spare jump seat, they finally decided that they just weren't allowed to do that anymore, he had to go...
So we had to find ourselves a new ground crew, return to the gate, and offload the guy. At this point we were getting close to the point where our flight crew were about to go illegal. We didn't pushback, for the second time, much ahead of the point where they'd have had to find us a new flight crew. Considering they probably couldn't do that the second pushback, and subsequent take off, was expedited. I was listening in on to the tower chatter on channel 9, and from the sounds of things we ducked in ahead of a fairly big queue, and rolled almost without a pause directly from the gate to the end of 32L and into the air.
That said, I arrived into Portland only two and a half hours later than planned, and I got to take a minor supporting role in an excellently produced farce. You can't complain too much...
Unlike last year the journey was fairly pleasant, just over two and a half hours in a straight run down the east coast main line from Leeds, where I was watching my wife graduate with her doctorate, instead of a torturous seven and a half hour journey across a flood torn west country.
However I've got a four thirty am check-in tomorrow morning, so right now it's time to get some sleep. More from me when I hit Portland...
Update: We took off late from Heathrow due to the late arrival of the equipment, we lost more time in the air due to adverse headwinds, and yet more time racked and stacked over a fog-bound O'Hare. So while I'm now on the ground in Chicago, I'm here two hours later than I should have been. Fortunately, for various values of fortunately, my trip through US immigration was relatively unscathed, and my onward flight turns out to be just as delayed going out as I was coming in. So I'm now camped out in the newly rennovated Red Carpet Club near B18. I'll let you know how things go when, or if, I make it into Portland later today...
Update: Finally made it into Portland. My connecting flight out of O'Hare was delayed, delayed again, and then delayed again. Then suddenly moved forward by an hour without warning, to a slot only a few minutes away. Apparently our plane was still sitting in Toronto, and United had decided to give us a different one at short notice. Presumably some other people were now without a plane, however...
This meant that I had to run across the terminal. Although I made it onto the flight ahead of most people, who no doubt had retired to the bar, or just taken a nap secure in the knowledge that had at least another hour to wait.
I also made it onto the plane before the flight crew, who took another hour and a half to arrive after we'd fully boarded. The cabin crew were looking distinctly nervous towards the end...
Once the flight crew arrived, did the safety check and the pre-fight, we'd closed the doors and started to pushback, one of the cabin crew noticed we had an extra passenger. The plane was full, every seat was taken, but there was still one poor guy still standing in the aisle clutching a valid boarding pass, but without a seat. After a brief discussion about whether he could sit in a spare jump seat, they finally decided that they just weren't allowed to do that anymore, he had to go...
So we had to find ourselves a new ground crew, return to the gate, and offload the guy. At this point we were getting close to the point where our flight crew were about to go illegal. We didn't pushback, for the second time, much ahead of the point where they'd have had to find us a new flight crew. Considering they probably couldn't do that the second pushback, and subsequent take off, was expedited. I was listening in on to the tower chatter on channel 9, and from the sounds of things we ducked in ahead of a fairly big queue, and rolled almost without a pause directly from the gate to the end of 32L and into the air.
That said, I arrived into Portland only two and a half hours later than planned, and I got to take a minor supporting role in an excellently produced farce. You can't complain too much...
Friday, July 11, 2008
The 2.0 firmware for the iPod touch
For Apple the events of the day are best described as a bit of a disaster, especially for iPod touch owners like myself who have been left in update limbo all day long.
However bypassing iTunes, which is still periodically claiming that the previous 1.1.4 firmware is the latest available, I managed to download the 2.0 firmware directly from the Apple servers and have now successfully upgraded my iPod to the new firmware.
Now I just have to push all my content back onto my iPod and I'm done...
Update: But at least I didn't have two cranky toddlers to cope with...
However bypassing iTunes, which is still periodically claiming that the previous 1.1.4 firmware is the latest available, I managed to download the 2.0 firmware directly from the Apple servers and have now successfully upgraded my iPod to the new firmware.
Now I just have to push all my content back onto my iPod and I'm done...
Update: But at least I didn't have two cranky toddlers to cope with...
Tuesday, July 08, 2008
Perl SAMP Alpha 3
I've just brought my pre-Trieste Perl Hub and test clients into line with the post-Trieste Working Draft document that Mark Taylor pushed to the SAMP mailing list at the tail end of June. Both the Perl Hub and the clients have been tested with Mark's Java SAMP kit pre-release and Luigi Paioro's SAMPY alpha 1 release. This release is being tagged alpha3, and is available under the terms of version 2 of the GPL license.
The new Perl Hub currently lacks
Apart from a quick update towards the end of the week to add
However if you want to get your feet dirty ahead of that, you'll need the following Perl modules installed:
As with my initial release of the prototype SAMP Hub back at the start of May, the code comes with no guarantees except there will be horrendous bugs. There is no documentation. Many of the Perl modules have inline POD, however most of it didn't keep up with the pace of development so it's out of date. However, once you install the additional modules, open up a terminal window and start the Hub as follows,
you can put the Hub through its paces by opening up two more terminal windows and running the testbed clients. You should start the listener client in all cases,
this is the test client that listens for
or
These two clients have a heart beat which will dispatch a call or a notification periodically. The first heartbeat will happen 15 seconds (or so) after the client has completed its registering with the Hub.
As always, feedback is appreciated...
Update: There is now a binary release of the Hub for x86_64 Linux and Intel Mac OSX. Just download the relevant zip file, unzip the executable, and run it. The dependencies, including Perl itself, should be taken care of using the magic of the Perl Packager.
Update: Apparently the x86_64 binary is fairly sensitive to installed library versions. This could just be my unfamiliarity with the packager. Reports of working, and non-working, installations would be good. Advice by people more familiar with
The new Perl Hub currently lacks
callAndWait( )
functionality, I'll be adding that in this week, but should in all other ways meet the 1.0 WD and be interoperable with the other emerging toolkits. Apart from a quick update towards the end of the week to add
callAndWait( )
, which will be tagged alpha4, this will be the last 'initial' release. After this I'll take some time to clean up the code, add some documentation, and make the client toolkit a bit more pleasant to use before making a 'first' release.However if you want to get your feet dirty ahead of that, you'll need the following Perl modules installed:
XMLRPC::Lite
(part of the SOAP::Lite
module), XML::Simple
, DateTime
, File::Spec
, Carp
, Data::Dumper
, Getopt::Long
, Socket
, Net::Domain
, POSIX
and Errno
. Depending on your version of Perl some, but not all, of these will ship with the core distribution. The rest can be obtained from CPAN. I think I've caught all the dependancies, but if you notice any I haven't listed any let me know I've forgotten them.As with my initial release of the prototype SAMP Hub back at the start of May, the code comes with no guarantees except there will be horrendous bugs. There is no documentation. Many of the Perl modules have inline POD, however most of it didn't keep up with the pace of development so it's out of date. However, once you install the additional modules, open up a terminal window and start the Hub as follows,
% tar -zxvf perl-samp-hub-alpha3.tar.gz
% cd perl-samp-hub-alpha3/
% ./samp_hub.pl
you can put the Hub through its paces by opening up two more terminal windows and running the testbed clients. You should start the listener client in all cases,
% tar -zxvf perl-samp-clients-alpha3.tar.gz
% cd perl-samp-clients-alpha3/
% ./listener_client.pl
this is the test client that listens for
notify( )
and call( )
's from the Hub. In the second window you can either start the client that exercises the notify( )
method, or the other than exercises the call( )
method. So% cd perl-samp-clients-alpha3/
% ./callAll_test.pl
or
% cd perl-samp-clients-alpha3/
% ./notifyAll_test.pl
These two clients have a heart beat which will dispatch a call or a notification periodically. The first heartbeat will happen 15 seconds (or so) after the client has completed its registering with the Hub.
As always, feedback is appreciated...
Update: There is now a binary release of the Hub for x86_64 Linux and Intel Mac OSX. Just download the relevant zip file, unzip the executable, and run it. The dependencies, including Perl itself, should be taken care of using the magic of the Perl Packager.
Update: Apparently the x86_64 binary is fairly sensitive to installed library versions. This could just be my unfamiliarity with the packager. Reports of working, and non-working, installations would be good. Advice by people more familiar with
PAR::Packer
than I am would be warmly welcomed.
Subscribe to:
Posts (Atom)