Adventures in Golang (Package registry)
OUTLINE
INTRODUCTION
I wanted to learn more about golang. I had used it in my previous job roles and I have been fascinated by the language. So I decided to deepen my knowledge of the language through doing projects.
One such project is a package registry. That every language that I have used so far (java, js, golang, ruby, python) have a place where you can get extra third-party libs, other than what it is normally included in the standard library of the language. That place is called a package registry. So in looking for one to implement, I stumbled on the npm registry. The api is here.
A package registry is basically a storage server with extra sauce (code). So basically a storage server provides clients (requesters), the means for uploading data to it and downloading data from it. So the npm client (the npm command) is all about interacting with the public npm registry or any registry that implements a subset or all of the methods of the api. For a simple private npm registry, we will support these commands when run by the client:
(a) npm install which does these actions when run on the client;
- getting the package json for a particular package from the registry and
- installing all the dependencies (first-party and third party dependencies from the pulled package.json)
(b) npm view. A sample run of the command is shown below
This command just retrieves the package.json from the registry and produces the output shown above.
(c) npm tag. A tag is an alias for a version of a package. This comamnd adds alias (tags) to a published version of a package. Basically, the non client sends these tags to the npm registry for those tags to be added to the package.json. This can be viewed by the npm view command.
(d) npm publish. This command actually uploads a version of a package to the registry after which the npm install command can be used to download the package.
(e) npm unpublish. This command deletes the package from the registry.
So what is a package according to npm? A package is described here and is:
1) a folder containing a program described by a package.json file
2) a gzipped tarball containing (1)
3) a url that resolves to (2)
4) a <name>@<version> that is published on the registry with (3)
5) a <name>@<tag> that points to (4)
6) a <name> that has a "latest" tag satisfying (5)
7) a <git remote url> that resolves to (1)
So for the private npm registry, we use @scope that is defined in the .npmrc file. The @scope is defined in the .npmrc as @scope:registry. This tells the npm client to resolve the url for the package to install to the value of the @scope:registry as shown below:
# in the .npmrc file
@scope:registry=<registry-url>
# npm command
npm install @scope/<package>
# is resovled by the npm client to
npm install <registry-url>/<package>
NOTES ON CREATING SERVERS IN GOLANG
In Go, an HTTP server relies on several interacting components: handlers, middleware, and a multiplexer( router ). When it includes all these parts, we call this server a web service.
First, the server’s multiplexer receives the client’s request. The multiplexer determines the destination for the request, then passes it along to the object capable of handling it. We call this object a handler. (The multiplexer itself is a handler that routes requests to the most appropriate handler.) Before the handler receives the request, the request may pass through one or more functions called middle-ware. Middleware changes the handlers’ behavior or performs auxiliary tasks, such as logging, authentication, or access control.
The mulitplexer here is an an object of type ServeMux defined in the http package. This ServeMux object contains, among other fields, a map data structure containing a mapping of the paths that you want your server to handle and the corresponding handler function.
A handler function must be of type func(http.ResponseWriter, *http.Request) ,where http.ResponseWriter and http.Request are two struct types defined in the net/http package. The object of type http.Request represents the incoming HTTP request and the http.ResponseWriter object is used to write back the response to the client making the request. The following is an example of a handler function:
func apiHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello World")
}
Anything you write to the ResponseWriter object, w , is sent back as a response to the client. Once you have written your handler function, the next step is to register it with a ServeMux object as shown below:
mux := http.NewServeMux()
mux.HandleFunc("/api", apiHandler)
An HTTP handler function accepts two parameters: a value of type http.ResponseWriter and a pointer to a value of type http.Request . The pointer object of http.Request type (defined in the net/http package) describes the incoming request. This type is also used to define an outgoing HTTP request.
A middleware function is defined as
func middleware(next http.Handler) http.Handler {
return func(w http.ResponseWriter, r *http.Request) {
// TODO: Execute our middleware logic here...
next.ServeHTTP(w, r)
}
}
NOTES ON CREATING THIS PROJECT
In creating this project, I learned certain things:
(a) In trying to organize my files, I came across this link and this link. These links describe how to organize your golang project, which is something like this
.
├── .git
├── cmd
| └── myapp
| └── main.go
├── examples
| └── example1
| └── main.go
├── internal
| ├── config
| | └── config.go
| └── store
| └── store.go
├── pkg
| ├── public1
| | └── public1.go
| └── public2
| └── public2.go
├── go.mod
└── go.sum
I had initially wanted to have the structure but it proved too daunting initially. So I had this folder structure instead
.
├── LICENSE.txt
├── Makefile
├── README.md
├── app.go
├── go.mod
├── go.sum
├── handler
│ ├── get.go
│ ├── publish.go
│ ├── tagdelete.go
│ ├── tagget.go
│ ├── tagput.go
│ └── tar.go
├── main.go
└── storage
└── fs.go
(b) I wanted to use the http.ServeMux but switched to gorilla/mux instead based on the following reasons (and based on the following flowchart):
- it does not support variables in Url paths
- it is very similar to http.ServeMux
- it is very well supported and archived.
(c) When testing the handlers, you do have to set an authToken like this
npm config set //<registry-url-without-the-protocol>/:_authToken <token>
Because if you do not set the authToken, the npm client asks you to run npm adduser which sets an authToken.
(d) If you want to pass something like config or a struct to the middleware or to the handlers,
# middleware
func middleware(s someStruct) func (http.HandlerFunc) http.HandlerFunc {
return func(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// TODO: Execute our middleware logic here...
next.ServeHTTP(w, r)
}
}
}
# handler
func apiHandler(s someStruct) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello World")
}
}