Writing microservices with Go Micro

This is a high level guide to writing microservices with go-micro.
If you want to learn more about microservices check out the introductory blog post here and if you want to know more about Micro, the microservice toolkit, look here.
Let’s get to it.

What is Go Micro?

Go Micro is a pluggable RPC based library which provides the fundamental building blocks for writing microservices in Go. The Micro philosophy is “batteries included” with a pluggable architecture. Out of the box, it implements service discovery using consul, communication via http and encoding using proto-rpc or json-rpc.
That’s a bit of a mouthful so let’s break it down.
Go Micro is:
  1. 1A library written in Go
  2. 2A set of pluggable interfaces
  3. 3RPC based
Go Micro provides interfaces for:
  1. 1Service Discovery
  2. 2Encoding
  3. 3Client/Server
  4. 4Pub/Sub
A more detailed breakdown can be found here.

Why Go Micro?

Go Micro started more than a year ago, initially serving a personal need. It was clear very soon after that it would be valuable to a broader audience also looking to write microservices. It’s based on experiences at various technology companies that operate microservice platforms at scale like Google and Hailo.
As mentioned before Go Micro is a pluggable architecture that focuses on providing Go based interfaces which when used together providing the building blocks for writing microservices. These interfaces can be satisfied by concrete implementations. For example the Registry interface for service discovery has a default implementation for Consul but can be swapped out for etcd, zookeeper, or anything else that is able to satisfy the interface.
The pluggable architecture means zero code rewriting if you want to swap out the underlying technologies.
Let’s get into writing a service.

Writing a service

If you want to get straight into reading the code, look at examples/service.
The top level Service interface is the main component for building a service. It wraps all the underlying packages of Go Micro into a single convenient interface.
type Service interface {
    Init(...Option)
    Options() Options
    Client() client.Client
    Server() server.Server
    Run() error
    String() string
}

1. Initialisation

A service is created like so using micro.NewService.
import "github.com/micro/go-micro"
service := micro.NewService()
Options can be passed in during creation.
service := micro.NewService(
 micro.Name("greeter"),
 micro.Version("latest"),
)
All the available options can be found here.
Go Micro also provides a way to set command line flags using micro.Flags.
import (
 "github.com/micro/cli"
 "github.com/micro/go-micro"
)
service := micro.NewService(
 micro.Flags(
  cli.StringFlag{
   Name:  "environment",
   Usage: "The environment",
  },
 )
)
To parse flags use service.Init. Additionally access flags use the micro.Action option.
service.Init(
 micro.Action(func(c *cli.Context) {
  env := c.StringFlag("environment")
  if len(env) > 0 {
   fmt.Println("Environment set to", env)
  }
 }),
)
Go Micro provides predefined flags which are set and parsed if service.Init is called. See all the flags here.

2. Defining the API

We use protobuf files to define the service API interface. This is a very convenient way to strictly define the API and provide concrete types for both the server and a client.
Here’s an example definition.
greeter.proto
syntax = "proto3";
service Greeter {
 rpc Hello(HelloRequest) returns (HelloResponse) {}
}
message HelloRequest {
 string name = 1;
}
message HelloResponse {
 string greeting = 2;
}
Here we’re defining a service handler called Greeter with the method Hello which takes the parameter HelloRequest type and returns HelloResponse.

3. Generate the API interface

We use protoc and protoc-gen-go to generate the concrete go implementation for this definition.
Go-micro uses code generation to provide client stub methods to reduce boiler plate code much like gRPC. It’s done via a protobuf plugin which requires a fork of golang/protobuf that can be found here github.com/micro/protobuf.
go get github.com/micro/protobuf/{proto,protoc-gen-go}
protoc --go_out=plugins=micro:. greeter.proto
The types generated can now be imported and used within a handler for a server or the client when making a request.
Here’s part of the generated code.
type HelloRequest struct {
 Name string `protobuf:"bytes,1,opt,name=name" json:"name,omitempty"`
}
type HelloResponse struct {
 Greeting string `protobuf:"bytes,2,opt,name=greeting" json:"greeting,omitempty"`
}
// Client API for Greeter service
type GreeterClient interface {
 Hello(ctx context.Context, in *HelloRequest, opts ...client.CallOption) (*HelloResponse, error)
}
type greeterClient struct {
 c           client.Client
 serviceName string
}
func NewGreeterClient(serviceName string, c client.Client) GreeterClient {
 if c == nil {
  c = client.NewClient()
 }
 if len(serviceName) == 0 {
  serviceName = "greeter"
 }
 return &greeterClient{
  c:           c,
  serviceName: serviceName,
 }
}
func (c *greeterClient) Hello(ctx context.Context, in *HelloRequest, opts ...client.CallOption) (*HelloResponse, error) {
 req := c.c.NewRequest(c.serviceName, "Greeter.Hello", in)
 out := new(HelloResponse)
 err := c.c.Call(ctx, req, out, opts...)
 if err != nil {
  return nil, err
 }
 return out, nil
}
// Server API for Greeter service
type GreeterHandler interface {
 Hello(context.Context, *HelloRequest, *HelloResponse) error
}
func RegisterGreeterHandler(s server.Server, hdlr GreeterHandler) {
 s.Handle(s.NewHandler(&Greeter{hdlr}))
}

4. Implement the handler

The server requires handlers to be registered to serve requests. A handler is an public type with public methods which conform to the signature func(ctx context.Context, req interface{}, rsp interface{}) error.
As you can see above, a handler signature for the Greeter interface looks like so.
type GreeterHandler interface {
        Hello(context.Context, *HelloRequest, *HelloResponse) error
}
Here’s an implementation of the Greeter handler.
import proto "github.com/micro/micro/examples/service/proto"
type Greeter struct{}
func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error {
 rsp.Greeting = "Hello " + req.Name
 return nil
}
The handler is registered with your service much like a http.Handler.
service := micro.NewService(
 micro.Name("greeter"),
)
proto.RegisterGreeterHandler(service.Server(), new(Greeter))
You can also create a bidirectional streaming handler but we’ll leave that for another day.

5. Running the service

The service can be run by calling server.Run. This causes the service to bind to the address in the config (which defaults to the first RFC1918 interface found and a random port) and listen for requests.
This will additionally Register the service with the registry on start and Deregister when issued a kill signal.
if err := service.Run(); err != nil {
 log.Fatal(err)
}

6. The complete service

greeter.go
package main
import (
        "log"
        "github.com/micro/go-micro"
        proto "github.com/micro/go-micro/examples/service/proto"
        "golang.org/x/net/context"
)
type Greeter struct{}
func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error {
        rsp.Greeting = "Hello " + req.Name
        return nil
}
func main() {
        service := micro.NewService(
                micro.Name("greeter"),
                micro.Version("latest"),
        )
        service.Init()
        proto.RegisterGreeterHandler(service.Server(), new(Greeter))
        if err := service.Run(); err != nil {
                log.Fatal(err)
        }
}
Note. The service discovery mechanism will need to be running so the service can register to be discovered by clients and other services. A quick getting started for that is here.

Writing a Client

The client package is used to query services. When you create a Service, a Client is included which matches the initialised packages used by the server.
Querying the above service is as simple as the following.
// create the greeter client using the service name and client
greeter := proto.NewGreeterClient("greeter", service.Client())
// request the Hello method on the Greeter handler
rsp, err := greeter.Hello(context.TODO(), &proto.HelloRequest{
 Name: "John",
})
if err != nil {
 fmt.Println(err)
 return
}
fmt.Println(rsp.Greeter)
proto.NewGreeterClient takes the service name and the client used for making requests.
The full example is can be found at examples/service.

Summary

Hopefully this blog post has been a helpful high level guide into writing microservices with Go Micro. You can find many more examples in the repo github.com/micro/examples to help you gain a further understanding of more real world solutions.

留言

張貼留言

這個網誌中的熱門文章

Json概述以及python對json的相關操作

圖片
什麼是json： JSON(JavaScript Object Notation) 是一種輕量級的數據交換格式。易於人閱讀和編寫。同時也易於機器解析和生成。它基於JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999的一個子集。JSON採用完全獨立於語言的文本格式，但是也使用了類似於C語言家族的習慣（包括C, C++, C#, Java, JavaScript, Perl, Python等）。這些特性使JSON成為理想的數據交換語言。 JSON建構於兩種結構： “名稱/值”對的集合（A collection of name/value pairs）。不同的語言中，它被理解為對象（object），紀錄（record），結構（struct），字典（dictionary），哈希表（hash table），有鍵列表（keyed list），或者關聯數組（associative array）。 值的有序列表（An ordered list of values）。在大部分語言中，它被理解為數組（array）。 這些都是常見的數據結構。事實上大部分現代計算機語言都以某種形式支持它們。這使得一種數據格式在同樣基於這些結構的編程語言之間交換成為可能。 jso官方說 ​​明參見：http://json.org/ Python操作json的標準api庫參考：http://docs.python.org/library/json.html 對簡單數據類型的encoding 和decoding： 使用簡單的json.dumps方法對簡單數據類型進行編碼，例如： import json obj = [[ 1 , 2 , 3 ], 123 , 123.123 , 'abc' ,{ 'key1' :( 1 , 2 , 3 ), 'key2' :( 4 , 5 , 6 )}] encodedjson = json.dumps(obj) print repr (obj) print encodedjson 輸出： [[1, 2, 3], 123, 123.123, 'abc', {'ke
閱讀完整內容

Docker容器日誌查看與清理

1. 問題 docker容器日誌導致主機磁盤空間滿了。 docker logs -f container_name 劈裡啪啦一大堆，很佔用空間，不用的日誌可以清理掉了。  2. 解決方法 2.1 找出Docker容器日誌 在linux上，容器日誌一般存放在 /var/lib/docker/containers/container_id/ 下面，查看各個日誌文件大小的腳本docker_log_size.sh，內容如下： #!/bin/sh echo "======== docker containers logs file size ========" logs=$(find /var/lib/docker/containers/ -name *-json.log) for log in $logs do ls -lh $log done # chmod +x docker_log_size.sh # ./docker_log_size.sh 2.2 清理Docker容器日誌（治標） 如果docker容器正在運行，那麼使用 rm -rf 方式刪除日誌後，通過 df -h 會發現磁盤空間並沒有釋放。 原因是在Linux或者Unix系統中，通過 rm -rf 或者文件管理器刪除文件，將會從文件系統的目錄結構上解除鏈接（unlink）。 如果文件是被打開的（有一個進程正在使用），那麼進程將仍然可以讀取該文件，磁盤空間也一直被佔用。 正確姿勢是 cat /dev/null > *-json.log ，當然你也可以通過 rm -rf 刪除後重啟docker。 接下來，提供一個日誌清理腳本 clean_docker_log.sh ，內容如下： #!/bin/sh echo "======== start clean docker containers logs ========" logs=$(find /var/lib/docker/containers/ -name *-json.log) for log in $logs d
閱讀完整內容

遠程控制管理工具ipmitool

圖片
IPMI（Intelligent Platform Management Interface）即智能平台管理接口是使硬件管理具備“智能化”的新一代通用接口標準。用戶可以利用IPMI 監視服務器的物理特徵，如溫度、電壓、電扇工作狀態、電源供應以及機箱入侵等。Ipmi 最大的優勢在於它是獨立於CPU BIOS 和OS 的，所以用戶無論在開機還是關機的狀態下，只要接通電源就可以實現對服務器的監控。 IPMI 是一種規範的標準，其中最重要的物理部件就是BMC(Baseboard Management Controller)，一種嵌入式管理微控制器，它相當於整個平台管理的“大腦”，通過它ipmi 可以監控各個傳感器的數據並記錄各種事件的日誌。 一般大廠的服務器都支持ipmi功能，比如DELL，HP，IBM。 此實驗選擇的硬件平台Dell R410 無idrac6卡，操作系統CentOS 5.6 1 .服務器啟動後，按'Ctrl + E'進入BMC管理工具界面 2 .設置'IPMI Over LAN'為On，啟用通過LAN方式的IPMI。 3 .選擇'IPMI LAN Channel'，進入設置IPMI的網絡屬性，IP Address '10.60.30.29'， Netmask '255.255.255.0' , Default Gateway '10.60.30.1'，與服務器eth0配置一致即可。 4 .選擇'User Configuration'，進入設置IPMI的訪問帳號, 'Admin account'保持'root'不變，'Enter Passwd'與'Confirm Passwd'填入'root' ipmitool命令使用 http://ipmitool.sourceforge.net/manpage.html 常用顯示本地網絡信息-I open表示接口本地 # ipmitool -I open lan print 1 Set in Progress : Set Complete Auth Type Suppo
閱讀完整內容