Category Archives: design

HEAD detached

You may have below after git log: commit f6601f5ae74d5b61303b17f1edcb56a6755c18ea Commit message commit 1e98a15d9a182ba965955bc960fa394a3de5c36a Commit message… git branch shows: 8c85904323f8:AlexaMacawDeciderService lipl$ git branch * (HEAD detached from refs/heads/lipl-branch-name) branch1 branch2 branch3 mainline Also it shows HEAD detached. It is good time to create a branch and specify most recent commit: git branch new-branch-name f6601f5ae74d5b61303b17f1edcb56a6755c18ea Thus, git branch shows… Read More »

Visitor pattern

1. Visitable public interface Visitable { int accept(Visitor visitor); } 2. ConcreteVisitable public class LiquorVisitable implements Visitable { @Override int accept(Visitor visitor) { return visitor.visit(this); } } public class FoodVisitable implements Visitable { @Override int accept(Visitor visitor) { return visitor.visit(this); } } 2. Visitor public interface Visitor { int visit(LiquorVisitable liquorVisitable); int visit(FoodVisitable foodVisitable); }… Read More »

2PC, 3PC

There are two basic distributed transaction protocols: two-phases-commit and three-phases-commit. two-phase-commit Assumption, coordinator and cohorts are never down. If coordinator is down, all other cohorts will be blocking and waiting for its recover. If a cohort is down and when it wakes up, it will get notified by coordinator(commit or rollback). Note: 2PC only guarantees… Read More »

Some feeling about GAE performance

A web service is deployed in GAE(google app engine). This is the loading test on GAE with different core, concurrency combinations. 1 core, 1 thread, 0.3s per reqeust 1 core, 160 threads, 12s per request 1 core, 150 threads, 8s per request 8 core, 50 threads, 5s per request 16 core, 50 threads, 3s per… Read More »

D3MAP example

I came across a project about d3map recently. The idea of the project is to parse the [longitude, latitude] of a location in USA, and show it on the USA map. Here are some example of this techniques: USA county map USA states map Circle Animation USA states with different color

Zookeeper Study Summary

In terms of CAP theorem, Zookeeper is a CP system. It emphasizes consistency in distributed system. It is a Leader – Follower based system. Zookeeper assumes that at least n/2+1 nodes are available. This guarantees brain-split won’t happen to zookeeper. How to handle partial failure? For example, there are 5 nodes. 2 of them are down.… Read More »

Kafka Study Summary

Some basic idea: A topic may has many partitions. A partition has to be fit in a server. Partitions are distributed and replicated over servers. Each partition has a leader server. If the leader server fails, then one of the follower will become leader. Partition, Segment: For a partition, it is physically stored by many… Read More »

zookeeper java.net.ConnectException: Connection refused

Today when I ran zookeeper by zkServer.sh start. It doesn’t run. Because I couldn’t see the port 2181 by netstat command. Then I tried zkServer.sh start-foreground. It showed error below: 2016-02-01 21:38:37,110 – FATAL [main:QuorumPeerMain@83] – Invalid config, exiting abnormally org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /home/hadoop/zookeeper/zookeeper-3.3.6/bin/../conf/zoo.cfg at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:110) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:99) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:76) Caused by: java.lang.IllegalArgumentException: /tmp/zookeeper/myid file is… Read More »

Use proxy_mod to redirect apache to tomcat

Recently, I moved my legacy blog PersonalPage to newer wordpress on Amazon ec2. WordPress is on apache php. Old peronalpage was wrote on tomcat. They are running on different ports. Today, I configured that I only visit through same port by using mod_proxy. By mod_proxy, all the request www.allenlipeng47.com/PersonalPage will forward to www.allenlipeng47.com:8080/PersonalPage In httpd, add below… Read More »

cap theorem

I’v been thinking such a problem for a long time. Let’s take my website for example. What if more and more people visit my website, and what do I do with my server. Natural way is that I pay more money to upgrade the CPU, memory or storage. But what if there are millions of visits that… Read More »