Monthly Archives: September 2020

nginx map, geo

below one will handle or not handle the request based on original country. geoip_country_code is a variable provided by nginx to tell the country of the IP.

# Allowed countries
#
map $geoip_country_code $allowed_country {
    default no;
    country_code_1 yes;
    country_code_2 yes;
}

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    # Disallow access based on GeoIP
    if ($allowed_country = no) {
        return 444;
    }

}

below one, it returns site content based on the host name. $host is a variable provided by nginx.

map $host $site_content {
  default "public";

  "grahamweldon.com" "grahamweldon";
  "slavitica.net" "slavitica";
}

http {
  [ ... ]

  server {
    listen 80 default_server;
    server_name _;
    location / {
      root /usr/local/websites/$site_content;
      try_files $uri index.html =404;
    }
  }
  
}

geo module is similar to map. But it can do complex mapping for IP. Below one store log based on client IP address.

geo $remote_addr $log_request {
  ranges;
  default 1;

  10.0.0.0-10.2.0.12 0;
  10.2.8.103-10.2.8.254 0;
}

location {
  root /some/path;
  access_log /var/log/access.log combined if=$log_request;
}

Category: web

nginx rate limiter

 

http {
   limit_req_zone $binary_remote_addr zone=mylimit:10m rate=2r/m;
   upstream myserver {
      server 127.0.0.1:8000;
   }

   server {
      listen 80;
      server_name myserver;
      location /api {
         limit_req mylimit;
         proxy_pass "http://myserver";
      }
   }
}

$binary_remote_addr, meaning the rate limiter is based on per IP. Each IP has 2 request per minute;
zone=mylimit:10m, define rate limiter name mylimit, use 10M to store it. According to Nginx doc, 1M stores 16K IPs.
rate = 2r/m, 2 request per minute for each IP

link1, link2

Category: web

Singleton

Singleton with public final field

Problem of this is that this can create another instance by reflection. The fix is that in private method, add a INSTANCE check.

public class SingletonWithPublicFinalField {

    public static final SingletonWithPublicFinalField INSTANCE = new SingletonWithPublicFinalField();

    private SingletonWithPublicFinalField() {
//        if (INSTANCE != null) {
//            throw new RuntimeException("Singleton instance already existed");
//        }
    }

    public static void main(String[] args) throws Exception{
        SingletonWithPublicFinalField instance1 = SingletonWithPublicFinalField.INSTANCE;
        System.out.println(instance1);

        Constructor.setAccessible(SingletonWithPublicFinalField.class.getConstructors(), true);
        Object instance2 = Class.forName(SingletonWithPublicFinalField.class.getName()).newInstance();
        System.out.println(instance2);
    }

}

Singleton with public static factory method. Good of it is that it is very like a class. It is easy to convert it into a normal non-singleton class. Just rewrite getInstance() method.

public class SingletonWithPublicStaticFactoryMethod {

    public static final SingletonWithPublicStaticFactoryMethod INSTANCE = new SingletonWithPublicStaticFactoryMethod();

    private SingletonWithPublicStaticFactoryMethod() {}

    public static SingletonWithPublicStaticFactoryMethod getInstance() {
        return INSTANCE;
    }

    public static void main(String[] args) throws Exception{
        SingletonWithPublicStaticFactoryMethod instance = SingletonWithPublicStaticFactoryMethod.getInstance();
        System.out.println(instance);
    }

}

Singleton serializable issue. When deserializing, it creates a different instance. The fix is to override readResolve() method and return the INSTANCE.

public class SingletonWithSerializable implements Serializable {

    public static final SingletonWithSerializable INSTANCE = new SingletonWithSerializable();

    private SingletonWithSerializable() {}

    public static SingletonWithSerializable getInstance() {
        return INSTANCE;
    }

    // When deserializing, it returns a different instance. The fix is to add below
//    private Object readRsolve() {
//        return INSTANCE;
//    }

    public static void main(String[] args) throws Exception{
        SingletonWithSerializable instance = SingletonWithSerializable.getInstance();
        System.out.println(instance);

        try (ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("file1.ser"))) {
            out.writeObject(instance);
        }

        try (ObjectInputStream in = new ObjectInputStream(new FileInputStream("file1.ser"))) {
            SingletonWithSerializable readObject = (SingletonWithSerializable) in.readObject();
            System.out.println(readObject);
        }
    }

}

The best option is to use enum singleton. It avoids reflection hacking or serializing issue very easily.

public enum  SingletonWithEnum {
    INSTANCE;

    public void doSomeMethod() {
        System.out.println("some method");
    }

    public static void main(String[] args) throws Exception {
        SingletonWithEnum singletonWithEnum = SingletonWithEnum.INSTANCE;
        SingletonWithEnum instance = SingletonWithEnum.INSTANCE;
        System.out.println(instance);

        try (ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("file1.ser"))) {
            out.writeObject(instance);
        }

        try (ObjectInputStream in = new ObjectInputStream(new FileInputStream("file1.ser"))) {
            SingletonWithEnum readObject = (SingletonWithEnum) in.readObject();
            System.out.println(readObject);
        }

        instance.doSomeMethod();
    }
    
}

 

Anti Pattern – Javabean Pattern

Call a non-argument constructor, then call a series of set method. Avoid using it.

class Person {
    String firstName;
    String lastName;
    int age;
    
    public Person() {}

    public String getFirstName() {
        return this.firstName;
    }
    
    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }
    
    public String getLastName() {
        return this.lastName;
    }
    
    public void setLastName(String lastName) {
        this.lastName = lastName;
    }
    
    public int getAge() {
        return this.age;
    }
    
    public void setAge(int age) {
        this.age = age;
    }

}

Anti Pattern: Telescoping constructor pattern

It lists all constructors. This is an anti pattern. Avoid using it.

class Person {
    String firstName;
    String lastName;
    int age;
    public Person() {}

    public Person(String firstName) {
        this.firstName = firstName;
    }

    public Person(String firstName, String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    public Person(String firstName, String lastName, int age) {
        this.firstName = firstName;
        this.lastName = lastName;
        this.age = age;
    }

}

 

static factory pattern

static factory method over constructors:
1. Constructor don’t have meaningful names, but static factory method has
2. Static factory method returns same type, subtype, primitives
3. Static factory could return same instance if needed

Sample of static factory method:

String value1 = String.valueOf(1);
String value2 = String.valueOf(1.0L);
Optional<String> value1 = Optional.empty();
Arrays.asList(1, 2, 3);

link

 

Bellman optimal equation for Q

 

q_function

Q(s, a), the expected return from starting state s, by taking action a at time t.
r(s, a), reward at state s, by taking action a
maxQ(s’, a’),  maximized expected return for next state-action(s’,a’). Need to find the a’, which maximizes it.

 

liner equation, non-linear equation

Single layer machine learning is actually linear equation. Linear equation is at most multi-variant polynomial equation, which at most include multiplication and adding.

Deep learning uses multi layers. The multi layers is considered to be non-linear equation.  Non-linear equation may include sin, cos, square etc on variances.

Both linear and non-linear equation fits curvature. However, non-linear is more flexible. Below are the equations for linear and non-linear ones.

Linear equations:

Y =\beta _{0} + \beta _{1}X_{1} + \beta _{2}X_{2} + \cdots + \beta _{k}X_{k}

Y =\beta _{0} + \beta _{1}X_{1} + \beta _{2}X_{1}^2

Non-linear equations:

\theta_{1} * X^\theta2

\theta_{1} + (\theta_{2}-\theta_{1}) * exp (-\theta_{3} * X ^\theta4)

\theta_{1} * cos(X + \theta_{4}) + \theta_{2} * cos (2 * X + \theta_{4}) + \theta_{3}

reference

 

Setup specific python version in virtualenv; setup virtualenv for PyCharm

  1. Download the python version you want, and install. https://www.python.org/downloads/
  2. In cmd, type python3.8
  3. Find the python3.8 Library path.
    virtualenv
  4. Create python3.8 virtualenv
    virtualenv -p /Library/Frameworks/Python.framework/Versions/3.7/bin/python3 venv
  5. venv2
  6. Activate venv
    source venv/bin/activate
  7. Check python version
    python --version

    venv3

  8. In PyCharm, select the created virtual env
    venv_pycharm
  9. In not existed in Project Interpreter, then click “Show All”, click ‘+’ to add a new one. Select the virtualenv that was just created.
    venv_pycharm2

nginx

install nginx

https://www.techrepublic.com/article/how-to-run-nginx-as-a-docker-container/

client_header_timeout, time when all header packets arrives nginx.
client_body_timeout, time when nginx hasn’t received a body packet, then timeout. NOT receiving whole body time. Kinda like read timeout.
send_timeout, similar to client_body_timeout. when nginx sends response to client, the time when nginx hasn’t received an ACK.
keepalive_timeout, connection becomes to idle when there is no traffic on it. keepalive_timeout destroys an idle connection when time arrives.

below will do longest prefix match. So if type http://localhost/blog. It will directs to www.allenlipeng47.com/blog url

server {

    location / {
        root /usr/share/nginx/html;
        index index.html index.htm;
    }

    location /blog {
        proxy_pass http://www.allenlipeng47.com;
    }
}