agentzh's home

Blogs

Tue, 06 Dec 2011 07:46:54 GMT

permalink

I'm posting nginx related stuffs and other things to my blog site on Blogger: http://agentzh.blogspot.com/

I'm also posting Nginx tutorials for both users and developers in Chinese at Sina Blog here: http://blog.sina.com.cn/openresty

Presentations

Tue, 29 Nov 2011 12:56:00 GMT

permalink

Here goes a list of slides that I used in various opensource events and school lectures.

These slides are powered by the slides.htm slide-making engine based on AJAX.

Note: Please use the arrow keys or pageup/pagedown keys on your keyboard to switch pages.

Year 2011

Applications of ngx_openresty and perl in lz.taobao.com

This talk was given at Beijing Perl Workshop 2011.

Watch the video online: http://v.ku6.com/show/TY8Vre59guTE_C8o.html

View in your web browser: http://agentzh.org/misc/slides/perl-lz-apps/

Download the slides: http://agentzh.org/misc/slides/perl-lz-apps.tar.gz

Year 2010

Introduction to nginx.conf scripting

This talk was given at the Beijing Perl Workshop 2010 April meeting and the Beijing OpenParty 2010 June event.

View in your web browser: http://agentzh.org/misc/slides/nginx-conf-scripting/

Download the slides: http://agentzh.org/misc/slides/nginx-conf-scripting.tar.gz

Please note that ngx_eval module is no longer recommended because we're in more favor of ngx_lua nowadays.

Recent developments in nginx.conf scripting

This talk was given at the Beijing OpenParty 2010 June event.

View in your web browser: http://agentzh.org/misc/slides/recent-dev-nginx-conf/

Download the slides: http://agentzh.org/misc/slides/recent-dev-nginx-conf.tar.gz

The state of the art of nginx.conf scripting

This talk was given at the ECUG 2010 event.

Watch the (Chinese-speech) video online: http://v.ku6.com/show/D00rqtnRwKzJdIsB.html

View the (English) slides in your web browser: http://agentzh.org/misc/slides/nginx-state-of-the-art/

Download the slides: http://agentzh.org/misc/slides/nginx-state-of-the-art.tar.gz

Year 2009

A Firefox cluster driven by JavaScript, Perl, & PL/PgSQL

This talk was given at the Beijing Perl Workshop 2009 Feb Meetup and Beijing OpenParty 2009 Feb event.

View the slides in your web browser: http://agentzh.org/misc/slides/BJPW200902/ffcluster.html

Introduction to our VDOM.pm & vdom-webkit cluster

Use vision information to do intelligent web scraping by rendering pages via a WebKit & VDOM cluster.

This talk was given at Beijing Perl Workshop 2009 and a meetup with Taobao.com's front-end engineers in the Hangzhou city.

View the slides in your browser: http://agentzh.org/misc/slides/taobao-fe/vdomwebkit.html

A Tour of Perl Testing

This talk was given at the Beijing Perl Workshop 2009 event.

View slides in your web browser: http://agentzh.org/misc/slides/BJPW2009/perl-testing.html

Year 2008

Nifty web apps on an OpenResty

This talk was given at the D2 Frontend Technology event organized by Yahoo! China.

View slides in your web browser: http://agentzh.org/misc/slides/openresty-d2/

OpenResty: Your Relational Cloud

OpenResty.pm is a general-purpose RESTful web service platform. In this talk I explained its origin, REST API, and its unusual applications.

This talk was given at the Beijing Perl Workshop 2008 event and the Yahoo! China Engineer Tech. Salon.

View slides in your web browser: http://agentzh.org/misc/slides/openresty-yes/

Year 2007

XUL::App - Jifty way of doing XUL

XUL::App is a nifty Firefox extension development framework based on Perl. It has a lot of parallels with Jifty. In particular, this framework allows you to build real-world modern Firefox extensions using Perl.

In this talk, I demonstrated how to use this framework to write a "hello, world" Firefox extension with I18N support in lines.

This talk was given at the Beijing Perl Workshop 2008 event.

View slides in your web browser: http://agentzh.org/misc/slides/xulapp/

Contribute to Pugs

Pugs is a Perl 6 compiler written in Haskell. In this talk, I introduced the Pugs project as well as my involvement in the automated testing work of the Perl 6 language to the PerlChina folks and encouraged people to contribute Pugs.

This talk was given at the Beijing Perl Workshop 2007 event.

View slides in your web browser: http://agentzh.org/misc/slides/contr_pugs/

Use UML::Class::Simple to generate simple UML class diagrams

UML::Class::Simple is my Perl CPAN module that can generate UML class diagrams automatically from arbitrary real-world Perl OO libraries.

This talk was given at the Beijing Perl Workshop 2007 event.

View slides in your web browser: http://agentzh.org/misc/slides/umlclass/

Year 2006

Introduction to XClips

XClips is an expert system development language designed by myself, which is implemented atop NASA's CLIPS. This talk explains the appearance of the language and some details of its implementation and applications.

This talk was given as a lecture at the Jiangsu University.

View slides in your web browser: http://agentzh.org/misc/slides/xclips/

The Art of Naming

This talk teaches the art of naming for programmers, using lots of examples from Perl 6, C/C++, Java, and other programming languages.

This talk was given as a lecture at the Jiangsu University.

View slides in your web browser:

XML in the real world

This talk shows various interesting applications of XML in the real world. It also introduces some of my web scraping tools based on Perl 5.

This talk was given as a lecture at the Jiangsu University.

View slides in your web browser: http://agentzh.org/misc/slides/xmlapp/

Functions and Methods

This lecture discusses functions and methods in Object-Oriented Programming (OOP). This is one of my series of lectures on OOP given at the Jiangsu University.

View slides in your web browser: http://agentzh.org/misc/slides/func/

Design Patterns - Using FAST

This talk explains the design patterns used in the construction process of the FAST (Flowchart Abstract Syntax Tree transformer) program, which can convert a program described by a flowchart into a structural program.

This talk was given as a lecture at the Jiangsu University.

View slides in your web browser: http://agentzh.org/misc/slides/patterns/

FollowMe

Wed, 16 Nov 2011 07:42:00 GMT

permalink

I'm quite active in Sina Weibo (A twitter clone very popular in China): http://weibo.com/agentzh

I do own a twitter account, @agentzh.

If you're interested in my opensource development work, go follow me on GitHub! http://github.com/agentzh/

MarkupPostBody

Mon, 25 Jul 2011 15:37:00 GMT

MarkupPostHead

Mon, 25 Jul 2011 15:37:00 GMT

ErlangFileReadLineBenchmark

Tue, 28 Jun 2011 09:28:00 GMT

permalink

Source listing

reader.erl

Here's the Erlang implementation A, not using Erlang binaries:

-module(reader).
-export([main/0]).

main() ->
    {ok, File} = file:open("data.log", [read]),
    Lines = count_lines(File, 0),
    io:format("Found ~w lines.~n", [Lines]),
    halt().

count_lines(File, Count) ->
    case file:read_line(File) of
        {ok, _Line} ->
            count_lines(File, Count + 1);
        eof ->
            Count
    end.

reader-bin.erl

Here's the Erlang implementation B, using Erlang binaries:

-module(reader).
-export([main/0]).

main() ->
    {ok, File} = file:open("data.log", [read, binary]),
    Lines = count_lines(File, 0),
    io:format("Found ~w lines.~n", [Lines]),
    halt().

count_lines(File, Count) ->
    case file:read_line(File) of
        {ok, _Line} ->
            count_lines(File, Count + 1);
        eof ->
            Count
    end.

reader-bin-raw.erl

Here's the Erlang implementation C, using Erlang binaries, raw file mode, and big read_ahead:

-module(reader).
-export([main/0]).

main() ->
    {ok, File} = file:open("data.log", [read, binary, raw, {read_ahead, 1024 * 1024}]),
    Lines = count_lines(File, 0),
    io:format("Found ~w lines.~n", [Lines]),
    halt().

count_lines(File, Count) ->
    case file:read_line(File) of
        {ok, _Line} ->
            count_lines(File, Count + 1);
        eof ->
            Count
    end.

reader.pl

Here's the Perl 5 implementation:

open my $in, "data.log" or die;
my $n;
while (<$in>) {
    $n++;
}
print "Found $n lines.\n";

Test data file

The data file is from a real nginx error.log file snapshot in our production environment. Here's its basic information:

$ wc data.log
    412003  12957676 197733977 data.log
$ ls -lh data.log
-rw-r--r-- 1 agentz users 189M 2011-06-28 15:08 data.log

Benchmark

To help OS cache the disk data, we only take the time of the programs' second runs.

Results on Slackware Linux x86_64

Linux Kernel: 2.6.33.1
CPU: Intel Core2Duo T9600
Hard disk: SATA 7200'
Erlang OTP: R14B01 (erts-5.8.2)
Perl: 5.12.1

The Erlang program A compiled in bytecode

$ erlc reader.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m14.023s
user	0m13.237s
sys	0m0.742s

The Erlang program A compiled in native code

$ erlc +native reader.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m14.007s
user	0m13.233s
sys	0m0.739s

The Erlang program B compiled in bytecode

$ erlc reader-bin.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m4.451s
user	0m4.322s
sys	0m0.104s

The Erlang program B compiled in native code

$ erlc +native reader-bin.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m4.531s
user	0m4.417s
sys	0m0.095s

The Erlang program C compiled in bytecode

$ erlc reader-bin-raw.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m10.001s
user	0m8.520s
sys	0m1.386s

The Erlang program C compiled in native code

$ erlc +native reader-bin-raw.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m10.398s
user	0m8.673s
sys	0m1.359s

The Perl 5 program

$ time perl reader.pl
Found 412003 lines.

real	0m0.332s
user	0m0.271s
sys	0m0.061s

Results on Mac OS X 10.6.7

CPU: Intel Core2Duo 1.86GHz
Hard disk: SSD
Erlang OTP: R14B02 (erts-5.8.3)
Perl: 5.12.3

The Erlang program A compiled in bytecode

$ erlc reader.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m32.254s
user	0m27.532s
sys	0m4.681s

The Erlang program A compiled in native code

$ erlc +native reader.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m32.043s
user	0m27.356s
sys	0m4.646s

The Erlang program B compiled in bytecode

$ erlc reader-bin.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m6.700s
user	0m6.447s
sys	0m0.212s

The Erlang program B compiled in native code

$ erlc +native reader-bin.erl
$ time erl -noshell -s reader main
Found 412003 lines.

real	0m6.721s
user	0m6.461s
sys	0m0.218s

The Perl 5 program

$ time perl reader.pl
Found 412003 lines.

real	0m0.677s
user	0m0.480s
sys	0m0.182s

Conclusion

Erlang's file:read_line/1 is painfully slow on both Linux and Mac OS X, as of the latest version R14. It's more than one order of magnitude slower than plain Perl 5. Enabling native code generation can make the Erlang version a little faster.

Update: Excluding the time involved in beam startup and quit only makes the result <100ms faster according to my tests on the Linux box.

AboutThisSite

Fri, 24 Jun 2011 03:01:00 GMT

This is site is powered by the excellent TiddlyWiki.

The site's source is hosted on GitHub: https://github.com/agentzh/agentzh.org

MainMenu

Fri, 24 Jun 2011 02:59:00 GMT

About me
Blogs
Follow me
Presentations
Projects
Contact me
About this site
RSS feed

Projects

Tue, 21 Jun 2011 08:04:00 GMT

permalink

The most important opensource project that I've been working on for years is OpenResty.

You can find most of my opensource projects on my GitHub homepage:

https://github.com/agentzh

Nginx

I've started or participated in the development of a bunch of nginx modules:

http://wiki.nginx.org/3rdPartyModules

Perl

I'm a Perl programmer that writes dozens of CPAN modules.

AboutMe

Tue, 21 Jun 2011 08:03:00 GMT

permalink

I'm a happy programmer living in Beijing, China. My real name is Zhang Yichun, or 章亦春 in Chinese.

I'm currently working as a Technical Expert for the Quantum Team in Taobao.com's Data Product & Platform department. I designed and implemented the underlying architecture of the front-end and middle-layer of the LineZing analytic application for Taobao.com's sellers.

The front-end application of LineZing is written completely in JavaScript and running in end user's web browser just like Gmail. The middle-layer is powered by ngx_openresty developed by ourselves, a massively enhanced version of nginx that serves as a full-fledged web app stack.

We've developed dozens of nginx C modules including ngx_echo, ngx_headers_more, ngx_lua, ngx_drizzle, and ngx_redis2, and opensourced them with the permission from the company. We do have users worldwide, including Taobao, Facebook, and Qunar.com.

I'm also leading the automated testing practices within the team, mainly with Perl 5.

In Taobao, I've created the (proprietary) LZSQL domain-spefic language to simplify the business logic coding involved in developing web services for data products. The LZSQL compiler is written in Perl 5 and emits compact Lua code. It has built-in capabilities to abstract distributed mysql cluster details and to perform complex SQL query optimizations that cannot be done by MySQL.

Prior to joining Taobao.com, I was working as a Senior Software Developer in Yahoo! China's Search Tech. Department.

While in Yahoo! China, I created OpenResty.pm to provide a generic service-oriented platform for the relavance search business, mainly written in Perl 5 & Haskell.

I designed and implemented Gecko/WebKit-powered intelligent webpage crawlers and extractors based on our own vision analysis and expert systems, main written in Perl 5 & C++. See my slides for details (use the arrow keys on your keyboard to switch pages).

As my first $work project, I created the SearchAll Firefox extension that does side-by-side search engine comparison, mainly written in JavaScript, which is also approved by Mozilla.

RSSFeed

Tue, 21 Jun 2011 07:42:00 GMT

permalink

This site's RSS feed is available at http://agentzh.org/index.xml .

DefaultTiddlers

Mon, 20 Jun 2011 09:29:00 GMT

AboutMe ContactMe

ContactMe

Mon, 20 Jun 2011 09:22:00 GMT

permalink

You're welcome to mail me via agentzh at gmail dot com. This mail address is also my MSN account.

You can usually find me via IRC on #nginx, #redis, #perl6, or #openresty at irc.freenode.net.

I'm quite active in the nginx and nginx-devel mailing lists. I'm also keeping an eye on the PerlChina, lua-l and erlang-questions mailing lists as well ;)

SiteUrl

Mon, 20 Jun 2011 09:19:00 GMT

http://agentzh.org/

SiteTitle

Fri, 17 Jun 2011 08:12:00 GMT

agentzh's home

SiteSubtitle

Fri, 17 Jun 2011 03:33:00 GMT

yet another idiot who loves programming

ZhangYichun

Tue, 14 Jun 2011 04:39:00 GMT

The owner of this site. He loves the nick agentzh though. But please do not capitalize the nick!