This book is an introductory usage guide to Node.js. It has two parts: one is practical "up and running" part with three chapters, the other is reference part with API, Data access, external modules etc five chapters.
In the first part, it gives quick introduction about Node.js, and how to install, how to build a chat server and twitter, also explains event loop, how to build robust node applications (patterns, error handling, and using multiple processors). Following the examples or code snippet, it is very easy to get node.js up and running. Also reader can get an introductory idea about what is node.js, how does it work, and what does it can do.
In the second part, it deep dives core APIs, helper APIs, Data Access, external modules, extending node. Event, http, I/O related APIs are core to build a web application using server side javascript. DNS, Crypto, and processes related help APIs are regularly useful for writing applications. In chapter 6, the books goes through a bunch of data stores from CouchDB, Redis, MongoDB, MySQL, PostgreSQL and also RabbitMQ. The last two chapters explore node modules, including popular Express (MVC framework), and Socket.IO (websocket), as well as extending node by building modules or creating packages.
nodejs.org has below definition:
Node.js
is a platform built on Chrome's
JavaScript runtime
for easily building fast, scalable network applications.
Node.js
uses an event-driven, non-blocking I/O model that makes it
lightweight and efficient, perfect for data-intensive real-time
applications that run across distributed devices.
Node.js
provides an unique opportunity for server side javascript to build
scalable web or network applications, using one language on both client
and server sides. High performance and high scalability application is the design goal of node, and its active community and extensive modules make programming javascript on server side very promising and attractive. After we deep understand Nodes' event-loop architecture, non-blocking I/O and event-driven model, using established design patterns, building node.js application will be a fun journey.
Next step, try out more node.js codes, and read Node.js in Action book.
Thursday, May 31, 2012
CSS position property
The CSS position property takes 5 values.
position: absolute
An absolute positioned element is positioned relative to the first parent element that has a position other than static. If no such element is found, the containing block is <html>. It is removed from normal flow. Sometimes we float columns with clear, instead of using relative position and absolute position.
position: relative
A relative positioned element is positioned relative to its normal flow, using top, right, bottom, and right. It also it establishes a context for position:absolute. It still occupies that original space in the normal flow.
position: fixed
An element with fixed position is positioned in relation to the browser window. It is removed from normal flow as absolute positioned element. It is common to see fixed position for twitter-like header, though IE6 needs some hack.
position: static
This is the default value if no CSS position specified. The element is not positioned, and is part of the normal flow of the page.
position: inherit
The element uses the position of its parent. The value from parent element position can be one of static, relative, absolute and fixed.
position: absolute
An absolute positioned element is positioned relative to the first parent element that has a position other than static. If no such element is found, the containing block is <html>. It is removed from normal flow. Sometimes we float columns with clear, instead of using relative position and absolute position.
position: relative
A relative positioned element is positioned relative to its normal flow, using top, right, bottom, and right. It also it establishes a context for position:absolute. It still occupies that original space in the normal flow.
position: fixed
An element with fixed position is positioned in relation to the browser window. It is removed from normal flow as absolute positioned element. It is common to see fixed position for twitter-like header, though IE6 needs some hack.
position: static
This is the default value if no CSS position specified. The element is not positioned, and is part of the normal flow of the page.
position: inherit
The element uses the position of its parent. The value from parent element position can be one of static, relative, absolute and fixed.
Wednesday, May 23, 2012
Viewport
The definitive article
If you want to understand Viewport, read this excellent article A tale of two viewports. The two viewports are layout viewport and visual viewport. The article has detailed explanation and diagram illustration, is well organized, informative, easy to understand. I took some notes here as my reading notes.
Two viewports
Viewport is exactly equal to the browser window. The viewport is not an HTML construct, so you cannot influence it by CSS. It just has the width and height of the browser window on desktop. On mobile, there are the visual viewport and the layout viewport. The visual viewport is the part of the page that’s currently shown on-screen. The layout viewport is considerably wider than the visual viewport. The CSS layout are calculated relative to the layout viewport.
CSS pixel and device pixel
CSS pixel indictates how your style sheet is rendered, and device pixel is used to measure user's screen which never changes. At zoom level 100% one CSS pixel is exactly equal to one device pixel. Zooming to 200% makes one CSS pixel grow to four times the size of one device pixels. If the user zooms in you get less available space in the window (so window inner width/height is decreasing).
screen.width and screen.height are measured in device pixel. They contain the total width and height of the user’s screen (never change).
window.innerWidth and window.innerHeight are measured in CSS pixel. They are the inner dimensions of the browser window, tell exactly how much space the user currently has available for the CSS layout. They include scroll bars.
window.pageXOffset and window.pageYOffset are measured in CSS pixel. They contain the horizontal and vertical scrolling offsets of the document.
document.documentElement.clientWidth and -Height always gives the viewport dimensions, regardless of the dimensions of the <html> element. They don't include scroll bars.
document.documentElement.offsetWidth and -Height gives the dimensions of <html> element.
So basically we have 5 different views, namely screen > window > viewport -> scrolling -> html
Viewport width on mobile
For layout viewport width, different browsers have different value. Safari iPhone uses 980px, Opera 850px, Android WebKit 800px, and IE 974px. The <html> element takes the width of the layout viewport initially, and CSS is interpreted as if the screen were significantly wider than the phone screen. This makes sure that your site’s layout behaves as it does on a desktop browser.
The layout viewport width is always the same. If you rotate your phone, the visual viewport changes, but the browser adapts to this new orientation by zooming in slightly so that the layout viewport is again as wide as the visual viewport. (?? don't fully understand this because layout viewport width is expected to be wider than visual viewport) Web developers don’t care about the height, only about the width.
document.documentElement.clientWidth and -Height contain the layout viewport’s dimensions.
window.innerWidth and window.innerHeight contain the visual viewport's dimensions. Obviously the measurements change when the user zooms out or in, since more or fewer CSS pixels fit into the screen.
Event coordinates
pageX/Y is still relative to the page in CSS pixels
clientX/Y is relative to the visual viewport in CSS pixels (?)
screenX/Y is relative to the screen in device pixels
Media Query
There are two relevant media queries: width/height and device-width/device-height.
Use width and forget device-width — on desktop
Meta viewport
<meta name="viewport" content="width=320">
If you want to understand Viewport, read this excellent article A tale of two viewports. The two viewports are layout viewport and visual viewport. The article has detailed explanation and diagram illustration, is well organized, informative, easy to understand. I took some notes here as my reading notes.
Two viewports
Viewport is exactly equal to the browser window. The viewport is not an HTML construct, so you cannot influence it by CSS. It just has the width and height of the browser window on desktop. On mobile, there are the visual viewport and the layout viewport. The visual viewport is the part of the page that’s currently shown on-screen. The layout viewport is considerably wider than the visual viewport. The CSS layout are calculated relative to the layout viewport.
CSS pixel and device pixel
CSS pixel indictates how your style sheet is rendered, and device pixel is used to measure user's screen which never changes. At zoom level 100% one CSS pixel is exactly equal to one device pixel. Zooming to 200% makes one CSS pixel grow to four times the size of one device pixels. If the user zooms in you get less available space in the window (so window inner width/height is decreasing).
screen.width and screen.height are measured in device pixel. They contain the total width and height of the user’s screen (never change).
window.innerWidth and window.innerHeight are measured in CSS pixel. They are the inner dimensions of the browser window, tell exactly how much space the user currently has available for the CSS layout. They include scroll bars.
window.pageXOffset and window.pageYOffset are measured in CSS pixel. They contain the horizontal and vertical scrolling offsets of the document.
document.documentElement.clientWidth and -Height always gives the viewport dimensions, regardless of the dimensions of the <html> element. They don't include scroll bars.
document.documentElement.offsetWidth and -Height gives the dimensions of <html> element.
So basically we have 5 different views, namely screen > window > viewport -> scrolling -> html
Viewport width on mobile
For layout viewport width, different browsers have different value. Safari iPhone uses 980px, Opera 850px, Android WebKit 800px, and IE 974px. The <html> element takes the width of the layout viewport initially, and CSS is interpreted as if the screen were significantly wider than the phone screen. This makes sure that your site’s layout behaves as it does on a desktop browser.
The layout viewport width is always the same. If you rotate your phone, the visual viewport changes, but the browser adapts to this new orientation by zooming in slightly so that the layout viewport is again as wide as the visual viewport. (?? don't fully understand this because layout viewport width is expected to be wider than visual viewport) Web developers don’t care about the height, only about the width.
document.documentElement.clientWidth and -Height contain the layout viewport’s dimensions.
window.innerWidth and window.innerHeight contain the visual viewport's dimensions. Obviously the measurements change when the user zooms out or in, since more or fewer CSS pixels fit into the screen.
Event coordinates
pageX/Y is still relative to the page in CSS pixels
clientX/Y is relative to the visual viewport in CSS pixels (?)
screenX/Y is relative to the screen in device pixels
Media Query
There are two relevant media queries: width/height and device-width/device-height.
Use width and forget device-width — on desktop
Meta viewport
<meta name="viewport" content="width=320">
Tuesday, May 22, 2012
WebSocket
One major part of HTML5 is connectivity, and WebSocket is the one will change web in terms of communication protocol. Wikipedia defines WebSocket as below:
WebSocket is a web technology providing for bi-directional, full-duplex communications channels over a single TCP connection. The WebSocket API is being standardized by the W3C, and the WebSocket protocol has been standardized by the IETF as RFC 6455.
We are looking for different ways to do server push instead of client polling. This near real time server push mechanism provides better user experience on the web. Comet (long polling) defines push technologies applied to Ajax (XHR) web applications, XMLSocket flash also provides the ability to create flash socket, BOSH/XMPP is often used in web chat application. Then comes to HTML5 which has Server sent event (one way), and WebSocket (two ways) for client/server bi-directional communication. The old frequent polling mechanism was out-dated which was a short-term solution before we have these new technologies.
WebSocket is becoming popular and broadly adopted on both server side and client side. Java, .NET, PHP, Python and server side Javascript etc all have WebSocket support. Recently I looked at node.js and tried out socket.io, which is pretty awesome to provide a WebSocket server. There are many discussion about Nginx reverse proxy support for WebSocket, so that deployment diagram can have Nginx (serving static resources) in front of Socket.IO (WebSocket). This comparison table has part of the list http://en.wikipedia.org/wiki/Comparison_of_WebSocket_implementations. Btw, Kaazing WebSocket Gateway is very famous due to HTML5 evangelist/trainer Peter Lubber from Kaazing. Another one is Union Platform which supports WebSocket too.
On the client side, there are many WebSocket libraries to support WebSocket programming. Using the native WebSocket API is hard (for now) due to browser support compatibility and the spec is not finalized. Old browsers don't have WebSocket support (fallback to Flash or Long Polling), newer browsers have unsecure WebSocket implementation, latest modern browsers start to implement WebSocket specification, and mobile browsers have different behaviors too. For browser WebSocket support compatibility, check out http://caniuse.com/#search=websocket With this kind of complexity or mess, we see more client libraries built for intermittent solutions. Usually the library has feature support detection, elegant fallback (polyfill), and native WebSocket API usage. Recently I looked at some Javascript libraries (Socket.IO client, SockJS and web-socket-js), and did read web-socket-js code which is use WebSocket API and fallback to Flash (WebSocketMain.swf). If you looked at socket.io, it has more protocol detection and fallback to cover more scenarios.
During the test using web-socket-js, I learnt Flash socket policy file configuration. Starting from Flash Player 9, there is a new strict access policy for Flash applications that make Socket or XMLSocket connections to a remote host. It now requires the presence of a socket policy file on the server. It means you have control to the server which your flash application connects to. There are two ways to deploy socket policy file: one is port 843 (not practical for production and firewall), the other is destination port (usually 80 or 443). In Nginx, we can set up the port 843 or use error page for 400 error as following
server {
listen 834;
server_name localhost;
location / {
rewrite ^(.*)$ /crossdomain.xml;
}
}
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
error_page 400 /crossdomain.xml;
}
The crossdomain.xml looks like:
<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
<allow-access-from domain="*" secure="false" to-ports="*"/>
<site-control permitted-cross-domain-policies="master-only" />
</cross-domain-policy>
Finally, don't forget to check out WebRTC from google, it enables real-time communication on browser based on HTML5 and Javascript. Google open source it to try to disrupt Microsoft skype and Apple Facetime.
WebSocket is a web technology providing for bi-directional, full-duplex communications channels over a single TCP connection. The WebSocket API is being standardized by the W3C, and the WebSocket protocol has been standardized by the IETF as RFC 6455.
We are looking for different ways to do server push instead of client polling. This near real time server push mechanism provides better user experience on the web. Comet (long polling) defines push technologies applied to Ajax (XHR) web applications, XMLSocket flash also provides the ability to create flash socket, BOSH/XMPP is often used in web chat application. Then comes to HTML5 which has Server sent event (one way), and WebSocket (two ways) for client/server bi-directional communication. The old frequent polling mechanism was out-dated which was a short-term solution before we have these new technologies.
WebSocket is becoming popular and broadly adopted on both server side and client side. Java, .NET, PHP, Python and server side Javascript etc all have WebSocket support. Recently I looked at node.js and tried out socket.io, which is pretty awesome to provide a WebSocket server. There are many discussion about Nginx reverse proxy support for WebSocket, so that deployment diagram can have Nginx (serving static resources) in front of Socket.IO (WebSocket). This comparison table has part of the list http://en.wikipedia.org/wiki/Comparison_of_WebSocket_implementations. Btw, Kaazing WebSocket Gateway is very famous due to HTML5 evangelist/trainer Peter Lubber from Kaazing. Another one is Union Platform which supports WebSocket too.
On the client side, there are many WebSocket libraries to support WebSocket programming. Using the native WebSocket API is hard (for now) due to browser support compatibility and the spec is not finalized. Old browsers don't have WebSocket support (fallback to Flash or Long Polling), newer browsers have unsecure WebSocket implementation, latest modern browsers start to implement WebSocket specification, and mobile browsers have different behaviors too. For browser WebSocket support compatibility, check out http://caniuse.com/#search=websocket With this kind of complexity or mess, we see more client libraries built for intermittent solutions. Usually the library has feature support detection, elegant fallback (polyfill), and native WebSocket API usage. Recently I looked at some Javascript libraries (Socket.IO client, SockJS and web-socket-js), and did read web-socket-js code which is use WebSocket API and fallback to Flash (WebSocketMain.swf). If you looked at socket.io, it has more protocol detection and fallback to cover more scenarios.
During the test using web-socket-js, I learnt Flash socket policy file configuration. Starting from Flash Player 9, there is a new strict access policy for Flash applications that make Socket or XMLSocket connections to a remote host. It now requires the presence of a socket policy file on the server. It means you have control to the server which your flash application connects to. There are two ways to deploy socket policy file: one is port 843 (not practical for production and firewall), the other is destination port (usually 80 or 443). In Nginx, we can set up the port 843 or use error page for 400 error as following
server {
listen 834;
server_name localhost;
location / {
rewrite ^(.*)$ /crossdomain.xml;
}
}
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
error_page 400 /crossdomain.xml;
}
The crossdomain.xml looks like:
<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
<allow-access-from domain="*" secure="false" to-ports="*"/>
<site-control permitted-cross-domain-policies="master-only" />
</cross-domain-policy>
Finally, don't forget to check out WebRTC from google, it enables real-time communication on browser based on HTML5 and Javascript. Google open source it to try to disrupt Microsoft skype and Apple Facetime.
Thursday, May 17, 2012
Image Optimization
Recently we did pngout for more than 2000 files, but few of them could not be optimized due to bit depth is 64 or null. However, we have not covered other image types like JPEG, GIF. There are a lot of discussions and talks about image optimization for web performance. In our project so far, we just started the first step, but we followed 80/20 rule to optimize our website.
Generally speaking, image optimization has 2 categories: lossless and lossy.
The safe way and easy way is to do lossless image optimization during build process and development phase. There are some high level guidelines for this optimization effort:
PNG or JPG?
Optimize PNG file
As some tools are PNG-specific, while others only optimize DEFLATE, in general one must use a combination of 2 tools in sequence for optimal compression: one which optimizes filters (and removes ancillary chunks), and one which optimizes DEFLATE. Most commonly, OptiPNG is used for the first (non-DEFLATE) step, and either of AdvanceCOMP or PNGOUT is used for the DEFLATE step. For more info or detailed comparison among PNG, JPEG, TIFF, check out http://en.wikipedia.org/wiki/Portable_Network_Graphics
Frequently used Tools
pngcrush
pngout
OptiPNG
ImageOptim
imgopt
Smush.it
Generally speaking, image optimization has 2 categories: lossless and lossy.
- Lossless Image Compression: Remove surplus info from image (Smush.it)
- Lossy Image Compression: Match image quality to screen resolution
The safe way and easy way is to do lossless image optimization during build process and development phase. There are some high level guidelines for this optimization effort:
- Don't use GIF unless you need animated GIF
- Crush PNG (pngout)
- Crush JPG (jpegtran)
- Know PNG24 to PNG8 Conversion Guidelines
- Know the time to choose JPG over PNG (Don't use PNG for everything)
- Combine images (csssprite/spriteMe)
- Compress favicon
- Crop and compress user uploaded avatar
PNG or JPG?
- PNG is a lossless compression format
- JPG doesn't support transparency
- PNG works best for vector type graphics with hard lines.
- JPG works best for anything with complex gradients like photo, screenshot etc
- Use imagemagick to batch convert PNG to JPG
Optimize PNG file
- color depth
Color depth can range from 1 to 64 bits per pixel, reduce color depth can reduce file size
- ancillary chunks
PNG supports metadata—remove it can reduce file size
- interlacing
As each pass of the Adam7 algorithm is separately filtered, this can increase file size
- filter
The choice of filter is potentially very variable and impacting file size
- compression
DEFLATE compressors can produce smaller files.
As some tools are PNG-specific, while others only optimize DEFLATE, in general one must use a combination of 2 tools in sequence for optimal compression: one which optimizes filters (and removes ancillary chunks), and one which optimizes DEFLATE. Most commonly, OptiPNG is used for the first (non-DEFLATE) step, and either of AdvanceCOMP or PNGOUT is used for the DEFLATE step. For more info or detailed comparison among PNG, JPEG, TIFF, check out http://en.wikipedia.org/wiki/Portable_Network_Graphics
Frequently used Tools
pngcrush
pngout
OptiPNG
ImageOptim
imgopt
Smush.it
Monday, May 14, 2012
Native Monetization
Today there is an article on TechCrunch called Five Ways Native Monetization Is Changing Silicon Valley to discuss native monetization. Jon Steinberg defines native monetization in his blog Native monetization as follows:
From Jon's blog and Wilson's keynote, I have some rough idea about native monetization.
Advertising is changing to more user friendly, experience oriented, and context relevance, instead of old ad banner way which was just dump the message without understanding user interest. News portal (popular website) in dot com era was the perfect place for ad banner due to its high number of visit per day. However, in web2.0 era, pulling is changing to pushing with Facebook news feed, RSS reader, twitter etc. More and more people don't go to specific news portal for daily news (I follow @cnnbrk on twitter for breaking news, follow techcrunch for tech company news). This ad channel is getting weak.
Starting from Google AdSense which displays relevance ad based on search result, new advertising changes the way to monetize your product. No matter it is inline image, audio, video or geolocation based advertisement, the promotion way is more nature, more experience oriented. There are more and more web2.0 sprees are moving towards native monetization. In my mind, native means content + advertising for user experience. Read user's mind is critical to go with native monetization.
Here are some examples without native monetization. DrawSomething free version has advertisement, but the full screen ad on mobile is not organic to the experience. Why it cannot display ad based on the guessing word? The result is less playing or blindly close it. Yahoo seems doing the same thing, when you read a news, the banner ad is kind of static, not relating to what you read (if user wants to spend time reading long article, s/he should be interested in the content, so displaying ad organic to the reading experience). The result is seeking other reading channel (ad free) or just never look at sidebar. Groupon, local advertisement, has already IPO, however I doubt its long term business model for two reasons. One is it is only venue based "native" monetization, not reading user's mind, and push coupons based on user's interests, lifestyle or purchase history. The other is too many competitors (google offer, amazonLocal, livingsocial) because its model is easy to replicate.
However, apart from facebook, twitter, google etc, amazon actually also is doing native monetization from its online purchase experience. Their advertising is more unique and nature to shopping experience. When you purchase something, amazon.com will recommend others related merchandise from your current purchase, based on your purchase history and other people's purchase history. Intelligence built in amazon shopping experience is to try to read shopper's mind.
I strongly believe native monetization is about what to display (reading user's mind) and how to display (multimedia). The ultimate goal of native is to let users like advertising.
Native monetization is deriving revenue from a platform in a way that is unique and organic to the experience. It’s the reason why Twitter, Facebook, Foursquare, and BuzzFeed avoid just slapping banners anywhere they can fit them. The theory behind these avoidances is that users will be happier with advertising or forms of revenue generation that fit into the context.Fred Wilson also talked about native monetization.
From Jon's blog and Wilson's keynote, I have some rough idea about native monetization.
Advertising is changing to more user friendly, experience oriented, and context relevance, instead of old ad banner way which was just dump the message without understanding user interest. News portal (popular website) in dot com era was the perfect place for ad banner due to its high number of visit per day. However, in web2.0 era, pulling is changing to pushing with Facebook news feed, RSS reader, twitter etc. More and more people don't go to specific news portal for daily news (I follow @cnnbrk on twitter for breaking news, follow techcrunch for tech company news). This ad channel is getting weak.
Starting from Google AdSense which displays relevance ad based on search result, new advertising changes the way to monetize your product. No matter it is inline image, audio, video or geolocation based advertisement, the promotion way is more nature, more experience oriented. There are more and more web2.0 sprees are moving towards native monetization. In my mind, native means content + advertising for user experience. Read user's mind is critical to go with native monetization.
Here are some examples without native monetization. DrawSomething free version has advertisement, but the full screen ad on mobile is not organic to the experience. Why it cannot display ad based on the guessing word? The result is less playing or blindly close it. Yahoo seems doing the same thing, when you read a news, the banner ad is kind of static, not relating to what you read (if user wants to spend time reading long article, s/he should be interested in the content, so displaying ad organic to the reading experience). The result is seeking other reading channel (ad free) or just never look at sidebar. Groupon, local advertisement, has already IPO, however I doubt its long term business model for two reasons. One is it is only venue based "native" monetization, not reading user's mind, and push coupons based on user's interests, lifestyle or purchase history. The other is too many competitors (google offer, amazonLocal, livingsocial) because its model is easy to replicate.
However, apart from facebook, twitter, google etc, amazon actually also is doing native monetization from its online purchase experience. Their advertising is more unique and nature to shopping experience. When you purchase something, amazon.com will recommend others related merchandise from your current purchase, based on your purchase history and other people's purchase history. Intelligence built in amazon shopping experience is to try to read shopper's mind.
I strongly believe native monetization is about what to display (reading user's mind) and how to display (multimedia). The ultimate goal of native is to let users like advertising.
Saturday, May 12, 2012
First error with node.js
This week I started to read "Node up and running" book, and followed the examples to explore node.js which is really promising for javascript on server side. Installing node, or setup other libraries using NPM is quite easy, and running sample node.js is also trivia. However, my ever first error from node.js is as below
Well, did a quick check ps aux | grep node found the previous node process was still there, but I did Ctrl-Z, right? So far I have not realized the root cause, so I went ahead to kill -9 node-PID to get rid of previous node process, and get the updated node.js running.
Why previous quite (Ctrl-Z) didn't work? Quickly checked wikipedia and found the answer
http://en.wikipedia.org/wiki/Signal_%28computing%29
It is clear that I should use Ctrl-C to terminate the node process, or kill the process.
node.js:201It is quite understandable from "listen EADDRINUSE", the error means some other process already occupy the address. But I did quit original node process.
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: listen EADDRINUSE
at errnoException (net.js:614:11)
at Array.0 (net.js:704:26)
at EventEmitter._tickCallback (node.js:192:40)
Well, did a quick check ps aux | grep node found the previous node process was still there, but I did Ctrl-Z, right? So far I have not realized the root cause, so I went ahead to kill -9 node-PID to get rid of previous node process, and get the updated node.js running.
Why previous quite (Ctrl-Z) didn't work? Quickly checked wikipedia and found the answer
http://en.wikipedia.org/wiki/Signal_%28computing%29
It is clear that I should use Ctrl-C to terminate the node process, or kill the process.
Tuesday, May 8, 2012
Increase initcwnd for TCP Performance
MTU & MSS
MTU (maximum transmission unit): Nearly all IP over Ethernet implementations use the Ethernet V2 frame format, which is 1500 bytes. Linux ifconfig output can show MTU size.
MSS (The maximum segment size) is a parameter of the TCP protocol that specifies the largest amount of data, specified in octets, that a computer or communications device can receive in a single TCP segment, and therefore in a single IP datagram. It does not count the TCP header or the IP header.
Therefore, TCP/IP Headers + MSS ≤ MTU
MTU = 1500
TCP Header = 20
IP Header = 20
TCP Option = 12 (optional)
MSS = 1460 (1448 if there is TCP Option)
Two TCP Windows
Congestion Window (cwnd) controls the number of packets a TCP flow may have in the network in any given time. cwnd is dynamically adapting to changing network condition. TCP Slow-start is one of the algorithms that TCP uses in its quest to control congestion inside the network and it is also known as the exponential growth phase.When TCP reaches a certain threshold (also known as sstrsesh) it will enter the linear growth, Congestion avoidance. Linux 2.6.39 increased the initial congestion window to 10 packets, previous versions are 3.
The Receiver Advertised Window (rwnd) is the buffer size sent in each ACK from TCP receiver to TCP sender. The window size is 65535 (64K) bytes on Windows/Mac/iOS.
The purpose of sliding window is to prevent from the sender to send too many packets to over flow the network resource or the receiver's buffer. The "sliding window size" is the maximum amount of data we can send without having to wait for ACK.
Therefore, with suggested 10 initcwnd size, sliding window can have 10 * MSS data flowing the network without ACK, it will definitely improve TCP performance, eliminating TCP slow start.
How to Change initcwnd/initrwnd
ip route show
sudo ip route change default via 192.168.1.1 dev eth0 initcwnd 10
ip route show
ip route show
sudo ip route change default via 192.168.1.1 dev eth0 initrwnd 10
ip route show
Notes:
1. The advertised receive window on Linux is called initrwnd. It can only be adjusted on linux kernel 2.6.33 and newer
2. This changes the initcwnd and initrwnd until the next reboot.
3. To persist the changes, try out below script (copied from cdnplanet.com, but not verified)
cp /etc/sysconfig/network-scripts/ifup-post /etc/sysconfig/network-scripts/ifup-post.bak; sed -i -e "/^exit 0/d" /etc/sysconfig/network-scripts/ifup-post; echo "ip route change " $(ip route show | grep '^default' | sed 's/initcwnd [0-9]*//') " initcwnd 10" >> /etc/sysconfig/network-scripts/ifup-post; echo "exit 0" >> /etc/sysconfig/network-scripts/ifup-post
Other Tunings
disable net.ipv4.tcp_slow_start_after_idle
setting tcp_slow_start_after_idle to 0 (for disabling it) to speed up initial connections, otherwise it will cause your keepalive connection to return to slow start after TCP_TIMEOUT_INIT (3 seconds)
disable Nagle algorithm
TCP implementations usually provide applications with an interface to disable the Nagle algorithm. This is typically called the TCP_NODELAY option.
Other options on CentOS
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_rmem=4096 87380 16777216
sysctl -w net.ipv4.tcp_wmem=4096 87380 16777216
sysctl -w net.core.netdev_max_backlog=30000
sysctl -w net.ipv4.tcp_congestion_control=htcp
Reference
http://kernelnewbies.org/Linux_2_6_39
http://www.cdnplanet.com/blog/tune-tcp-initcwnd-for-optimum-performance/
http://monolight.cc/2010/12/increasing-tcp-initial-congestion-window/
http://www.osischool.com/protocol/Tcp/slidingWindow/index.php
MTU (maximum transmission unit): Nearly all IP over Ethernet implementations use the Ethernet V2 frame format, which is 1500 bytes. Linux ifconfig output can show MTU size.
MSS (The maximum segment size) is a parameter of the TCP protocol that specifies the largest amount of data, specified in octets, that a computer or communications device can receive in a single TCP segment, and therefore in a single IP datagram. It does not count the TCP header or the IP header.
Therefore, TCP/IP Headers + MSS ≤ MTU
MTU = 1500
TCP Header = 20
IP Header = 20
TCP Option = 12 (optional)
MSS = 1460 (1448 if there is TCP Option)
Two TCP Windows
Congestion Window (cwnd) controls the number of packets a TCP flow may have in the network in any given time. cwnd is dynamically adapting to changing network condition. TCP Slow-start is one of the algorithms that TCP uses in its quest to control congestion inside the network and it is also known as the exponential growth phase.When TCP reaches a certain threshold (also known as sstrsesh) it will enter the linear growth, Congestion avoidance. Linux 2.6.39 increased the initial congestion window to 10 packets, previous versions are 3.
The Receiver Advertised Window (rwnd) is the buffer size sent in each ACK from TCP receiver to TCP sender. The window size is 65535 (64K) bytes on Windows/Mac/iOS.
The purpose of sliding window is to prevent from the sender to send too many packets to over flow the network resource or the receiver's buffer. The "sliding window size" is the maximum amount of data we can send without having to wait for ACK.
Therefore, with suggested 10 initcwnd size, sliding window can have 10 * MSS data flowing the network without ACK, it will definitely improve TCP performance, eliminating TCP slow start.
How to Change initcwnd/initrwnd
ip route show
sudo ip route change default via 192.168.1.1 dev eth0 initcwnd 10
ip route show
ip route show
sudo ip route change default via 192.168.1.1 dev eth0 initrwnd 10
ip route show
Notes:
1. The advertised receive window on Linux is called initrwnd. It can only be adjusted on linux kernel 2.6.33 and newer
2. This changes the initcwnd and initrwnd until the next reboot.
3. To persist the changes, try out below script (copied from cdnplanet.com, but not verified)
cp /etc/sysconfig/network-scripts/ifup-post /etc/sysconfig/network-scripts/ifup-post.bak; sed -i -e "/^exit 0/d" /etc/sysconfig/network-scripts/ifup-post; echo "ip route change " $(ip route show | grep '^default' | sed 's/initcwnd [0-9]*//') " initcwnd 10" >> /etc/sysconfig/network-scripts/ifup-post; echo "exit 0" >> /etc/sysconfig/network-scripts/ifup-post
Other Tunings
disable net.ipv4.tcp_slow_start_after_idle
setting tcp_slow_start_after_idle to 0 (for disabling it) to speed up initial connections, otherwise it will cause your keepalive connection to return to slow start after TCP_TIMEOUT_INIT (3 seconds)
disable Nagle algorithm
TCP implementations usually provide applications with an interface to disable the Nagle algorithm. This is typically called the TCP_NODELAY option.
Other options on CentOS
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_rmem=4096 87380 16777216
sysctl -w net.ipv4.tcp_wmem=4096 87380 16777216
sysctl -w net.core.netdev_max_backlog=30000
sysctl -w net.ipv4.tcp_congestion_control=htcp
Reference
http://kernelnewbies.org/Linux_2_6_39
http://www.cdnplanet.com/blog/tune-tcp-initcwnd-for-optimum-performance/
http://monolight.cc/2010/12/increasing-tcp-initial-congestion-window/
http://www.osischool.com/protocol/Tcp/slidingWindow/index.php
Monday, May 7, 2012
Javascript Logging
Quick Reference
console.log(object[, object, ...])
console.debug(object[, object, ...])
console.info(object[, object, ...])
console.warn(object[, object, ...])
console.error(object[, object, ...])
console.assert(expression[, object, ...])
console.dir(object)
console.dirxml(node)
console.trace()
console.group(object[, object, ...])
console.groupEnd()
console.time(name)
console.timeEnd(name)
console.profile([title])
console.profileEnd()
console.count([title])
Examples
console.log("My name is %s, and I am %d years old", 'Jim', 30);
Cross Browsers
Firefox (Firebug/FireBug Lite)
Chrome
Safari
Opera
IE
Option 1:
window.console||(window.console={log:function(){}});
Option 2:
window.log = function(){
log.history = log.history || []; // store logs to an array for reference
log.history.push(arguments);
if(this.console){
if (arguments.length == 1) {
console.log(arguments[0]);
}
else {
console.log( Array.prototype.slice.call(arguments) );
}
}
};
Reference
http://geekswithblogs.net/renso/archive/2009/07/02/firebug-console-quick-reference-guide.aspx
http://getfirebug.com/logging
http://paulirish.com/2009/log-a-lightweight-wrapper-for-consolelog/
console.log(object[, object, ...])
console.debug(object[, object, ...])
console.info(object[, object, ...])
console.warn(object[, object, ...])
console.error(object[, object, ...])
console.assert(expression[, object, ...])
console.dir(object)
console.dirxml(node)
console.trace()
console.group(object[, object, ...])
console.groupEnd()
console.time(name)
console.timeEnd(name)
console.profile([title])
console.profileEnd()
console.count([title])
Examples
console.log("My name is %s, and I am %d years old", 'Jim', 30);
| String Substitution Patterns | |
| %s | String |
| %d, %i | Integer (numeric formatting is not yet supported) |
| %f | Floating point number (numeric formatting is not yet supported) |
| %o | Object hyperlink |
Cross Browsers
Firefox (Firebug/FireBug Lite)
Chrome
Safari
Opera
IE
Option 1:
window.console||(window.console={log:function(){}});
Option 2:
window.log = function(){
log.history = log.history || []; // store logs to an array for reference
log.history.push(arguments);
if(this.console){
if (arguments.length == 1) {
console.log(arguments[0]);
}
else {
console.log( Array.prototype.slice.call(arguments) );
}
}
};
Reference
http://geekswithblogs.net/renso/archive/2009/07/02/firebug-console-quick-reference-guide.aspx
http://getfirebug.com/logging
http://paulirish.com/2009/log-a-lightweight-wrapper-for-consolelog/
Subscribe to:
Posts (Atom)