Kubernetes1.5源碼分析(二) apiServer之資源註冊

源碼版本

Kubernetes v1.5.0

簡介

k8s裏面有各類資源，如Pod、Service、RC、namespaces等資源，用戶操做的其實也就是這一大堆資源。但這些資源並非雜亂無章的，使用了GroupVersion的方式組織在一塊兒。每一種資源都屬於一個Group，而資源還有版本之分，如v一、v1beta1等。
k8s目前正在使用的API groups：

• "core" group：它的REST path是api/v1

• "extensions" group：它的REST path是/apis/extensions/v1beta1

• "autoscaling", "abac" ...

服務器的URL的格式：/prefix/group/version/... (例如：/apis/extensions/v1beta1)

重要結構體

APIGroupVersion：對API資源的組織，裏面包含了Storage、GroupVersion、Mapper、Serializer、Convertor等成員。Storage是etcd的接口，這是一個map類型，每一種資源都會與etcd創建一個鏈接；GroupVersion表示該APIGroupVersion屬於哪一個Group、哪一個version；Serializer用於序列化，反序列化；Convertor提供各個不一樣版本進行轉化的接口；Mapper實現了RESTMapper接口。

type APIGroupVersion struct {
    // key存在對象的url，value是一個rest.Storage，用於對接etcd存儲
    Storage map[string]rest.Storage
    // 該group的prefix，例如核心組的Root是'/api'
    Root string

    // 包含相似'api/v1'這樣的string，用於標識這個實例
    GroupVersion unversioned.GroupVersion

    // OptionsExternalVersion controls the Kubernetes APIVersion used for common objects in the apiserver
    // schema like api.Status, api.DeleteOptions, and api.ListOptions. Other implementors may
    // define a version "v1beta1" but want to use the Kubernetes "v1" internal objects. If
    // empty, defaults to GroupVersion.
    OptionsExternalVersion *unversioned.GroupVersion

    // 關鍵性成員
    Mapper meta.RESTMapper

    // 對象序列化和反序列化器
    Serializer     runtime.NegotiatedSerializer
    ParameterCodec runtime.ParameterCodec

    // 如下4個都是被賦值爲Scheme結構
    Typer     runtime.ObjectTyper
    Creater   runtime.ObjectCreater
    // 相互轉換任意api版本的對象，須要事先註冊轉換函數
    Convertor runtime.ObjectConvertor
    Copier    runtime.ObjectCopier

    Linker    runtime.SelfLinker

    // 用於訪問許可控制
    Admit   admission.Interface
    Context api.RequestContextMapper

    MinRequestTimeout time.Duration

    // SubresourceGroupVersionKind contains the GroupVersionKind overrides for each subresource that is
    // accessible from this API group version. The GroupVersionKind is that of the external version of
    // the subresource. The key of this map should be the path of the subresource. The keys here should
    // match the keys in the Storage map above for subresources.
    SubresourceGroupVersionKind map[string]unversioned.GroupVersionKind

    // ResourceLister is an interface that knows how to list resources
    // for this API Group.
    ResourceLister APIResourceLister
}

APIGroupVersion的建立接口是pkg/genericapiserver/genericapiserver.go中的newAPIGroupVersion()接口，在接口在建立APIGroupVersion還用到了好幾個別的結構：APIGroupInfo、Scheme、GroupMeta。下面一個一個介紹：
APIGroupInfo:

type APIGroupInfo struct {
    // 該Group的元信息
    GroupMeta apimachinery.GroupMeta
    // 不一樣版本的全部的Storage
    VersionedResourcesStorageMap map[string]map[string]rest.Storage
    // OptionsExternalVersion controls the APIVersion used for common objects in the
    // schema like api.Status, api.DeleteOptions, and api.ListOptions. Other implementors may
    // define a version "v1beta1" but want to use the Kubernetes "v1" internal objects.
    // If nil, defaults to groupMeta.GroupVersion.
    // TODO: Remove this when https://github.com/kubernetes/kubernetes/issues/19018 is fixed.
    OptionsExternalVersion *unversioned.GroupVersion

    // core group的話，對應的就是api.Scheme
    Scheme *runtime.Scheme
    // NegotiatedSerializer controls how this group encodes and decodes data
    NegotiatedSerializer runtime.NegotiatedSerializer
    // ParameterCodec performs conversions for query parameters passed to API calls
    ParameterCodec runtime.ParameterCodec

    // 全部resources信息,key就是resource的path
    // 好比：key爲"replicationcontrollers/scale",GroupVersionKind: autoscaling, v1, Scale
    SubresourceGroupVersionKind map[string]unversioned.GroupVersionKind
}

Scheme: 用於API資源之間的序列化、反序列化、版本轉換。Scheme裏面還有好幾個map，前面的結構體存儲的都是unversioned.GroupVersionKind、unversioned.GroupVersion這些東西，這些東西本質上只是表示資源的字符串標識，Scheme存儲了對應着標誌的具體的API資源的結構體，即relect.Type

type Scheme struct {
    // versionMap allows one to figure out the go type of an object with
    // the given version and name.
    gvkToType map[unversioned.GroupVersionKind]reflect.Type

    // typeToGroupVersion allows one to find metadata for a given go object.
    // The reflect.Type we index by should *not* be a pointer.
    typeToGVK map[reflect.Type][]unversioned.GroupVersionKind

    // unversionedTypes are transformed without conversion in ConvertToVersion.
    unversionedTypes map[reflect.Type]unversioned.GroupVersionKind

    // unversionedKinds are the names of kinds that can be created in the context of any group
    // or version
    // TODO: resolve the status of unversioned types.
    unversionedKinds map[string]reflect.Type

    // Map from version and resource to the corresponding func to convert
    // resource field labels in that version to internal version.
    fieldLabelConversionFuncs map[string]map[string]FieldLabelConversionFunc

    // defaulterFuncs is an array of interfaces to be called with an object to provide defaulting
    // the provided object must be a pointer.
    defaulterFuncs map[reflect.Type]func(interface{})

    // converter stores all registered conversion functions. It also has
    // default coverting behavior.
    converter *conversion.Converter

    // cloner stores all registered copy functions. It also has default
    // deep copy behavior.
    cloner *conversion.Cloner
}

GroupMeta： 主要包括Group的元信息，裏面的成員RESTMapper，與APIGroupVersion同樣，其實APIGroupVersion的RESTMapper直接取之於GroupMeta的RESTMapper.一個Group可能包含多個版本，存儲在GroupVersion中，而GroupVersion是默認存儲在etcd中的版本。

type GroupMeta struct {
    // 默認版本
    GroupVersion unversioned.GroupVersion

    // 該Group中可能會有多個版本，該字段就包含了全部的versions
    GroupVersions []unversioned.GroupVersion

    // 用於編解碼
    Codec runtime.Codec

    // SelfLinker can set or get the SelfLink field of all API types.
    // TODO: when versioning changes, make this part of each API definition.
    // TODO(lavalamp): Combine SelfLinker & ResourceVersioner interfaces, force all uses
    // to go through the InterfacesFor method below.
    SelfLinker runtime.SelfLinker

    // 用於類型，對象之間的轉換
    RESTMapper meta.RESTMapper

    // InterfacesFor returns the default Codec and ResourceVersioner for a given version
    // string, or an error if the version is not known.
    // TODO: make this stop being a func pointer and always use the default
    // function provided below once every place that populates this field has been changed.
    InterfacesFor func(version unversioned.GroupVersion) (*meta.VersionInterfaces, error)

    // InterfacesByVersion stores the per-version interfaces.
    InterfacesByVersion map[unversioned.GroupVersion]*meta.VersionInterfaces
}

RESTMapper： 用於管理全部對象的信息。外部要獲取的話，直接經過version,group獲取到RESTMapper，而後經過kind類型能夠獲取到對應的信息。該RESTMapper實際上是實現了一個DefaultRESTMapper結構。

type DefaultRESTMapper struct {
    defaultGroupVersions []unversioned.GroupVersion

    resourceToKind       map[unversioned.GroupVersionResource]unversioned.GroupVersionKind
    kindToPluralResource map[unversioned.GroupVersionKind]unversioned.GroupVersionResource
    kindToScope          map[unversioned.GroupVersionKind]RESTScope
    singularToPlural     map[unversioned.GroupVersionResource]unversioned.GroupVersionResource
    pluralToSingular     map[unversioned.GroupVersionResource]unversioned.GroupVersionResource

    interfacesFunc VersionInterfacesFunc

    // aliasToResource is used for mapping aliases to resources
    aliasToResource map[string][]string
}

APIRegistrationManager：這個結構體主要提供了已經"registered"的概念，將全部已經註冊的，已經激活的，第三方的的GroupVersions進行了彙總，還包括了各個GroupVersion的GroupMeta(元數據)。

type APIRegistrationManager struct {
    // 因此已經registered的GroupVersions
    registeredVersions map[unversioned.GroupVersion]struct{}

    // 第三方註冊的GroupVersions,這些都向apiServer動態註冊的
    thirdPartyGroupVersions []unversioned.GroupVersion

    // 全部已經enable的GroupVersions，能夠經過EnableVersions()將要enable的GroupVersion加入進來。
    // 只有enable了，才能使用對應的GroupVersion
    enabledVersions map[unversioned.GroupVersion]struct{}

    // 全部groups的GroupMeta
    groupMetaMap map[string]*apimachinery.GroupMeta
    // 跟環境變量'KUBE_API_VERSIONS'有關
    envRequestedVersions []unversioned.GroupVersion
}

APIRegistrationManager初始化

該結構的路徑：pkg/apimachinery/registered/registered.go
在該文件中咱們能看到初始化了一個DefaultAPIRegistrationManager對象：

var (
    DefaultAPIRegistrationManager = NewOrDie(os.Getenv("KUBE_API_VERSIONS"))
)

進入NewOrDie()接口看下：

func NewOrDie(kubeAPIVersions string) *APIRegistrationManager {
    m, err := NewAPIRegistrationManager(kubeAPIVersions)
    if err != nil {
        glog.Fatalf("Could not construct version manager: %v (KUBE_API_VERSIONS=%q)", err, kubeAPIVersions)
    }
    return m
}

func NewAPIRegistrationManager(kubeAPIVersions string) (*APIRegistrationManager, error) {
    m := &APIRegistrationManager{
        registeredVersions:      map[unversioned.GroupVersion]struct{}{},
        thirdPartyGroupVersions: []unversioned.GroupVersion{},
        enabledVersions:         map[unversioned.GroupVersion]struct{}{},
        groupMetaMap:            map[string]*apimachinery.GroupMeta{},
        envRequestedVersions:    []unversioned.GroupVersion{},
    }

    // 若是環境變量KUBE_API_VERSIONS進行了設置的話，進行遍歷
    if len(kubeAPIVersions) != 0 {
        // 經過逗號進行分隔
        for _, version := range strings.Split(kubeAPIVersions, ",") {
            // 解析version並轉換成GroupVersion格式
            // 通常這裏的version是group/version格式，好比'/api/v1'
            gv, err := unversioned.ParseGroupVersion(version)
            if err != nil {
                return nil, fmt.Errorf("invalid api version: %s in KUBE_API_VERSIONS: %s.",
                    version, kubeAPIVersions)
            }
            // 而後將該gv放入envRequestedVersions
            m.envRequestedVersions = append(m.envRequestedVersions, gv)
        }
    }
    // 不然返回一個空的APIRegistrationManager
    return m, nil
}

瞅了下咱們正在使用的環境，沒有配置KUBE_API_VERSIONS,即返回了一個空的結構，還提供了好多方法。

var (
    ValidateEnvRequestedVersions  = DefaultAPIRegistrationManager.ValidateEnvRequestedVersions
    AllPreferredGroupVersions     = DefaultAPIRegistrationManager.AllPreferredGroupVersions
    RESTMapper                    = DefaultAPIRegistrationManager.RESTMapper
    GroupOrDie                    = DefaultAPIRegistrationManager.GroupOrDie
    AddThirdPartyAPIGroupVersions = DefaultAPIRegistrationManager.AddThirdPartyAPIGroupVersions
    IsThirdPartyAPIGroupVersion   = DefaultAPIRegistrationManager.IsThirdPartyAPIGroupVersion
    RegisteredGroupVersions       = DefaultAPIRegistrationManager.RegisteredGroupVersions
    IsRegisteredVersion           = DefaultAPIRegistrationManager.IsRegisteredVersion
    IsRegistered                  = DefaultAPIRegistrationManager.IsRegistered
    Group                         = DefaultAPIRegistrationManager.Group
    EnabledVersionsForGroup       = DefaultAPIRegistrationManager.EnabledVersionsForGroup
    EnabledVersions               = DefaultAPIRegistrationManager.EnabledVersions
    IsEnabledVersion              = DefaultAPIRegistrationManager.IsEnabledVersion
    IsAllowedVersion              = DefaultAPIRegistrationManager.IsAllowedVersion
    EnableVersions                = DefaultAPIRegistrationManager.EnableVersions
    RegisterGroup                 = DefaultAPIRegistrationManager.RegisterGroup
    RegisterVersions              = DefaultAPIRegistrationManager.RegisterVersions
    InterfacesFor                 = DefaultAPIRegistrationManager.InterfacesFor
)

在分析apiServer的啓動流程的時候，你會發現初始化ServerRunOptions對象時，用到了好多上面的變量，好比：
路徑：pkg/genericapiserver/options/server_run_options.go

func NewServerRunOptions() *ServerRunOptions {
    return &ServerRunOptions{
        AdmissionControl:                         "AlwaysAdmit",
。。。
        // 這裏就使用了AllPreferredGroupVersions接口
        DefaultStorageVersions:                   registered.AllPreferredGroupVersions(),
。。。
        StorageVersions:                          registered.AllPreferredGroupVersions(),
    }
}

上面就使用到了registered.AllPreferredGroupVersions()接口,順便看下接口具體實現：

func (m *APIRegistrationManager) AllPreferredGroupVersions() string {
    // 若是沒有註冊groupMeta的話，這裏就==0
    // 不過不可能沒有註冊，至於在哪裏進行註冊就得看下後面介紹的GroupMeta初始化了
    if len(m.groupMetaMap) == 0 {
        return ""
    }
    var defaults []string
    for _, groupMeta := range m.groupMetaMap {
        defaults = append(defaults, groupMeta.GroupVersion.String())
    }
    sort.Strings(defaults)
    return strings.Join(defaults, ",")
}

該接口比較簡單，就是從m.groupMetaMap中取出全部的groupMeta，而後經過逗號拼接成"group1/version1,group2/version2,..."的字符串。

這裏能夠想一下，既然有list，那總得有groupMeta啊。而咱們看APIRegistrationManager的初始化，若是沒有設置KUBE_API_VERSIONS環境變量的話，根本就沒有groupMeta。
既然不可能沒有groupMeta，那確定得從別的地方進行register & enable。咱們能夠從APIRegistrationManager提供的RegisterGroup方法入手：

func (m *APIRegistrationManager) RegisterGroup(groupMeta apimachinery.GroupMeta) error {
    groupName := groupMeta.GroupVersion.Group
    if _, found := m.groupMetaMap[groupName]; found {
        return fmt.Errorf("group %v is already registered", m.groupMetaMap)
    }
    m.groupMetaMap[groupName] = &groupMeta
    return nil
}

該RegisterGroup接口的入參就是GroupMeta,因此咱們得繼續查看該結構的初始化了。

GroupMeta初始化

k8s現階段，API一共分爲13個Group：Core、apps、authentication、authorization、autoscaling、batch、certificates、componentconfig、extensions、imagepolicy、policy、rbac、storage。其中Core的Group Name爲空，它包含的API是最核心的API,如Pod、Service等，它的代碼位於pkg/api下面，其它12個Group代碼位於pkg/apis。每一個目錄下都有一個install目錄，裏面有一個install.go文件，接着經過init()負責初始化。這些程序都是經過下列文件進行import：
路徑： pkg/master/import_known_versions.go

package master

// These imports are the API groups the API server will support.
import (
    "fmt"

    _ "k8s.io/kubernetes/pkg/api/install"
    "k8s.io/kubernetes/pkg/apimachinery/registered"
    _ "k8s.io/kubernetes/pkg/apis/apps/install"
    _ "k8s.io/kubernetes/pkg/apis/authentication/install"
    _ "k8s.io/kubernetes/pkg/apis/authorization/install"
    _ "k8s.io/kubernetes/pkg/apis/autoscaling/install"
    _ "k8s.io/kubernetes/pkg/apis/batch/install"
    _ "k8s.io/kubernetes/pkg/apis/certificates/install"
    _ "k8s.io/kubernetes/pkg/apis/componentconfig/install"
    _ "k8s.io/kubernetes/pkg/apis/extensions/install"
    _ "k8s.io/kubernetes/pkg/apis/imagepolicy/install"
    _ "k8s.io/kubernetes/pkg/apis/policy/install"
    _ "k8s.io/kubernetes/pkg/apis/rbac/install"
    _ "k8s.io/kubernetes/pkg/apis/storage/install"
)

一共import了13個group。其中"k8s.io/kubernetes/pkg/api/install"就是Core Group，咱們就以它爲例，查看下對應的install.go文件。
路徑： pkg/api/install/install.go

var availableVersions = []unversioned.GroupVersion{v1.SchemeGroupVersion}

func init() {
    // 進行Versions註冊，其實就是存入APIRegistrationManager.registeredVersions中
    registered.RegisterVersions(availableVersions)
    externalVersions := []unversioned.GroupVersion{}
    for _, v := range availableVersions {
        // 判斷下是否已經註冊，並追加成一個切片
        if registered.IsAllowedVersion(v) {
            externalVersions = append(externalVersions, v)
        }
    }
    if len(externalVersions) == 0 {
        glog.V(4).Infof("No version is registered for group %v", api.GroupName)
        return
    }

    // 再進行enable，其實就是存入APIRegistrationManager.enabledVersions
    if err := registered.EnableVersions(externalVersions...); err != nil {
        glog.V(4).Infof("%v", err)
        return
    }
    // 該接口比較關鍵，進行單獨介紹
    if err := enableVersions(externalVersions); err != nil {
        glog.V(4).Infof("%v", err)
        return
    }
}

首先定義了一個切片availableVersions，裏面只有一個元素v1.SchemeGroupVersion:

const GroupName = ""
var SchemeGroupVersion = unversioned.GroupVersion{Group: GroupName, Version: "v1"}

根據該元素的定義，能夠看出availableVersions就定義了一個GroupName爲空，Version是'v1'的GroupVersion。接着把該GroupVersion放入APIRegistrationManager的registeredVersions和enabledVersions中。
registered的幾個接口實現比較簡單不進行介紹了，可是執行的enableVersions()是重頭戲，咱們繼續深刻：

func enableVersions(externalVersions []unversioned.GroupVersion) error {
    // 字面意思：將全部的Versions添加到Scheme
    // 又牽扯到Scheme，後面會介紹Scheme的初始化
    // 越深刻看牽扯出的概念越多，該接口也很重要，須要耐心層層挖掘
    addVersionsToScheme(externalVersions...)
    // 將一個GroupVersion做爲默認的,即'/api/v1'
    preferredExternalVersion := externalVersions[0]

    // 就是這裏！ 進行了GroupMeta的初始化。這就是咱們這小節要看的關鍵
    groupMeta := apimachinery.GroupMeta{
        GroupVersion:  preferredExternalVersion,
        GroupVersions: externalVersions,
        // RESTMapper也是關鍵所在，下面也會單作一節進行介紹
        RESTMapper:    newRESTMapper(externalVersions),
        SelfLinker:    runtime.SelfLinker(accessor),
        InterfacesFor: interfacesFor,
    }

    // 前面都是register和enable了versions，這裏纔是進行了Group的register
    // 該接口其實就是以第一個GroupVersion的groupName爲key，groupMeta爲value
    // 對APIRegistrationManager的groupMetaMap,進行了賦值
    if err := registered.RegisterGroup(groupMeta); err != nil {
        return err
    }
    return nil
}

到這步，咱們再結合以前APIRegistrationManager的初始化，就能知道groupMetaMap中應該有了好幾組groupMeta。那在ServerRunOptions對象初始化中調用的registered.AllPreferredGroupVersions()接口,能返回好幾個DefaultStorageVersions，至少確定有'/api/v1'。至於別的groupMeta，須要再看下別的install.go，大同小異就不展開一個一個講了。

groupMeta的初始化雖然結束了，可是這裏又引出一個關鍵Scheme，那麼繼續下一小節吧。。

Scheme初始化

在上一節介紹enableVersions()函數時，第一行即是調用了addVersionsToScheme(externalVersions...),將GroupVersions加到Scheme。咱們就來看下該接口:

func addVersionsToScheme(externalVersions ...unversioned.GroupVersion) {
    // add the internal version to Scheme
    if err := api.AddToScheme(api.Scheme); err != nil {
        // Programmer error, detect immediately
        panic(err)
    }
    // add the enabled external versions to Scheme
    for _, v := range externalVersions {
        if !registered.IsEnabledVersion(v) {
            glog.Errorf("Version %s is not enabled, so it will not be added to the Scheme.", v)
            continue
        }
        switch v {
        case v1.SchemeGroupVersion:
            if err := v1.AddToScheme(api.Scheme); err != nil {
                // Programmer error, detect immediately
                panic(err)
            }
        }
    }
}

接口中咱們能夠看到AddToScheme(api.Scheme)都是將GroupVersion加入到api.Scheme。咱們先將上面的接口解析放放，先看下api.Scheme是如何初始化的：
路徑：pkg/api/register.go

var Scheme = runtime.NewScheme()

定義了Scheme，再看NewScheme()：
路徑：pkg/runtime/scheme.go

func NewScheme() *Scheme {
    // 定義空的Scheme
    s := &Scheme{
        gvkToType:        map[unversioned.GroupVersionKind]reflect.Type{},
        typeToGVK:        map[reflect.Type][]unversioned.GroupVersionKind{},
        unversionedTypes: map[reflect.Type]unversioned.GroupVersionKind{},
        unversionedKinds: map[string]reflect.Type{},
        cloner:           conversion.NewCloner(),
        fieldLabelConversionFuncs: map[string]map[string]FieldLabelConversionFunc{},
        defaulterFuncs:            map[reflect.Type]func(interface{}){},
    }
    // 建立converter，用於不一樣版本對象轉換
    s.converter = conversion.NewConverter(s.nameFunc)
    // 增長一些轉換函數
    s.AddConversionFuncs(DefaultEmbeddedConversions()...)

    // Enable map[string][]string conversions by default
    if err := s.AddConversionFuncs(DefaultStringConversions...); err != nil {
        panic(err)
    }
    if err := s.RegisterInputDefaults(&map[string][]string{}, JSONKeyMapper, conversion.AllowDifferentFieldTypeNames|conversion.IgnoreMissingFields); err != nil {
        panic(err)
    }
    if err := s.RegisterInputDefaults(&url.Values{}, JSONKeyMapper, conversion.AllowDifferentFieldTypeNames|conversion.IgnoreMissingFields); err != nil {
        panic(err)
    }
    return s
}

上面就建立了一個空的Scheme。
知道哪裏建立Scheme後，咱們繼續回到上面的addVersionsToScheme()函數。
其實主要就是看兩個接口: api.AddToScheme()和v1.AddToScheme()。
先看第一個：

var (
    SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes, addDefaultingFuncs)
    AddToScheme   = SchemeBuilder.AddToScheme
)

經過runtime.NewSchemeBuilder()接口傳入兩個函數，而後建立了SchemeBuilder：

type SchemeBuilder []func(*Scheme) error

func (sb *SchemeBuilder) Register(funcs ...func(*Scheme) error) {
    for _, f := range funcs {
        *sb = append(*sb, f)
    }
}


func NewSchemeBuilder(funcs ...func(*Scheme) error) SchemeBuilder {
    var sb SchemeBuilder
    sb.Register(funcs...)
    return sb
}

根據上面的定義和函數能夠看出，SchemeBuilder就是一個接口切片，包含了addKnownTypes, addDefaultingFuncs兩個接口。
SchemeBuilder定義好了以後，繼續看AddToScheme:

func (sb *SchemeBuilder) AddToScheme(s *Scheme) error {
    for _, f := range *sb {
        if err := f(s); err != nil {
            return err
        }
    }
    return nil
}

該函數就是調用了addKnownTypes, addDefaultingFuncs兩個接口,咱們一個一個看：

func addKnownTypes(scheme *runtime.Scheme) error {
    if err := scheme.AddIgnoredConversionType(&unversioned.TypeMeta{}, &unversioned.TypeMeta{}); err != nil {
        return err
    }
    // 把下列對象加入到Scheme中
    // 該SchemeGroupVersion的GroupName爲空，Version是"__internal"
    // 因此該接口實際上是把k8s內置的version添加到Scheme，並且每一個group都有該步
    scheme.AddKnownTypes(SchemeGroupVersion,
        &Pod{},
        &PodList{},
        &PodStatusResult{},
        &PodTemplate{},
        &PodTemplateList{},
        &ReplicationControllerList{},
        &ReplicationController{},
        &ServiceList{},
        &Service{},
        &ServiceProxyOptions{},
        &NodeList{},
        &Node{},
        &NodeProxyOptions{},
        &Endpoints{},
        &EndpointsList{},
        &Binding{},
        &Event{},
        &EventList{},
        &List{},
        &LimitRange{},
        &LimitRangeList{},
        &ResourceQuota{},
        &ResourceQuotaList{},
        &Namespace{},
        &NamespaceList{},
        &ServiceAccount{},
        &ServiceAccountList{},
        &Secret{},
        &SecretList{},
        &PersistentVolume{},
        &PersistentVolumeList{},
        &PersistentVolumeClaim{},
        &PersistentVolumeClaimList{},
        &DeleteOptions{},
        &ListOptions{},
        &PodAttachOptions{},
        &PodLogOptions{},
        &PodExecOptions{},
        &PodProxyOptions{},
        &ComponentStatus{},
        &ComponentStatusList{},
        &SerializedReference{},
        &RangeAllocation{},
        &ConfigMap{},
        &ConfigMapList{},
    )

    // 在GroupName爲空，Version爲"v1"的groupVersion中，添加這些對象到Scheme
    scheme.AddUnversionedTypes(Unversioned,
        &unversioned.ExportOptions{},
        &unversioned.Status{},
        &unversioned.APIVersions{},
        &unversioned.APIGroupList{},
        &unversioned.APIGroup{},
        &unversioned.APIResourceList{},
    )
    return nil
}

查看AddKnownTypes()接口：

func (s *Scheme) AddKnownTypes(gv unversioned.GroupVersion, types ...Object) {
    if len(gv.Version) == 0 {
        panic(fmt.Sprintf("version is required on all types: %s %v", gv, types[0]))
    }
    for _, obj := range types {
        t := reflect.TypeOf(obj)
        if t.Kind() != reflect.Ptr {
            panic("All types must be pointers to structs.")
        }
        t = t.Elem()
        if t.Kind() != reflect.Struct {
            panic("All types must be pointers to structs.")
        }

        gvk := gv.WithKind(t.Name())
        s.gvkToType[gvk] = t
        s.typeToGVK[t] = append(s.typeToGVK[t], gvk)
    }
}

該接口主要操做了s.gvkToType和s.typeToGVK，用於轉換的目的。
綜上得出，是將internal version添加到Scheme中。
爲何會有一個internal version呢？ 其實每個Group都有一個internal version。而apiserver操做的也都是internal version.
舉個例子：假若有一個建立Pod的請求來了，apiserver首先會將請求給反序列化，用戶發過來的Pod請求每每是有版本的，好比v1，所以會反序列化爲一個v1.Pod。apiserver會當即將這個v1.Pod利用convertor轉換成internal.Pod，而後進行一些操做，最後要把它存到etcd裏面去，etcd裏面的Pod信息是有版本的，所以會先發生一次轉換，將其轉換爲v1.Pod，而後序列化存入etcd。
這樣看上去好像畫蛇添足？其實這就是k8s對api多版本的支持，這樣用戶能夠以一個v1beta1建立一個Pod,而後存入etcd的是一個相對穩定的版本，好比v1版本。

internal version添加完成後，繼續回到最開始的addVersionsToScheme()函數，還要繼續執行v1.AddToScheme(api.Scheme)函數.其實就是把v1版本的api添加到Scheme中，和添加internal版本同樣。
咱們看看v1.AddToScheme。
路徑：pkg/api/v1/register.go

var (
    SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes, addDefaultingFuncs, addConversionFuncs, addFastPathConversionFuncs)
    AddToScheme   = SchemeBuilder.AddToScheme
)

這裏能夠看到v1相比較internal版本，還多了好幾個函數addConversionFuncs, addFastPathConversionFuncs。
這些函數在執行AddToScheme()時其實都會要遍歷執行，能夠深刻看下。其實就是向Scheme添加了轉換函數，好比將v1.Pod轉換爲internal.Pod，將internal.Pod轉換爲v1.Pod。若是同時有v1,v2,v3會如何進行轉換？其實也仍是先統一轉換成internal，而後再轉換爲相應的版本(v1,v2,v3).因此internal至關於轉換的橋樑，更好的支持了不一樣版本的api。

到這裏Scheme的初始化基本結束了。 上面講GroupMeta初始化時還引出了關鍵性的RESTMapper，因此繼續進行介紹。

RESTMapper初始化

該部分的初始化就直接看GroupMeta初始化時調用的接口newRESTMapper():
路徑： pkg/api/install/install.go

func newRESTMapper(externalVersions []unversioned.GroupVersion) meta.RESTMapper {
    // 這些是API最頂層的對象，能夠理解爲沒有namespace的對象
    // 根據有無namespace，對象分爲兩類：RESTScopeNamespace和RESTScopeRoot
    rootScoped := sets.NewString(
        "Node",
        "Namespace",
        "PersistentVolume",
        "ComponentStatus",
    )

    // 須要忽略Scheme中以下的kinds
    ignoredKinds := sets.NewString(
        "ListOptions",
        "DeleteOptions",
        "Status",
        "PodLogOptions",
        "PodExecOptions",
        "PodAttachOptions",
        "PodProxyOptions",
        "NodeProxyOptions",
        "ServiceProxyOptions",
        "ThirdPartyResource",
        "ThirdPartyResourceData",
        "ThirdPartyResourceList")

    mapper := api.NewDefaultRESTMapper(externalVersions, interfacesFor, importPrefix, ignoredKinds, rootScoped)

    return mapper
}

其實全部的api資源能夠分爲兩類：一類是有namespace，另外一類是沒有namespace。好比該接口中的Node、Namespace、PersistentVolume、ComponentStatus不屬於任何namespace。ignoredKinds是下面接口須要用到的參數，表示遍歷Scheme時忽略這些kinds。
而後調用api.NewDefaultRESTMapper()，importPrefix參數爲："k8s.io/kubernetes/pkg/api"，
interfacesFor是一個接口。
路徑：pkg/api/mapper.go

func NewDefaultRESTMapper(defaultGroupVersions []unversioned.GroupVersion, interfacesFunc meta.VersionInterfacesFunc,
    importPathPrefix string, ignoredKinds, rootScoped sets.String) *meta.DefaultRESTMapper {
    // 加入Scheme,並繼續調用下面的接口
    return NewDefaultRESTMapperFromScheme(defaultGroupVersions, interfacesFunc, importPathPrefix, ignoredKinds, rootScoped, Scheme)
}

func NewDefaultRESTMapperFromScheme(defaultGroupVersions []unversioned.GroupVersion, interfacesFunc meta.VersionInterfacesFunc,
    importPathPrefix string, ignoredKinds, rootScoped sets.String, scheme *runtime.Scheme) *meta.DefaultRESTMapper {
    // 初始化了一個DefaultRESTMapper對象
    mapper := meta.NewDefaultRESTMapper(defaultGroupVersions, interfacesFunc)
    // 根據輸入的defaultGroupVersions,好比"/api/v1"，從Scheme中遍歷全部的kinds
    // 而後進行Add
    for _, gv := range defaultGroupVersions {
        for kind, oType := range scheme.KnownTypes(gv) {
            gvk := gv.WithKind(kind)
            // 過濾掉不屬於"k8s.io/kubernetes/pkg/api"路徑下的api，和ignoredKinds
            if !strings.Contains(oType.PkgPath(), importPathPrefix) || ignoredKinds.Has(kind) {
                continue
            }
            // 判斷該kind是否有namespace屬性
            scope := meta.RESTScopeNamespace
            if rootScoped.Has(kind) {
                scope = meta.RESTScopeRoot
            }
            // 而後將該gvk加入到對應的組中
            mapper.Add(gvk, scope)
        }
    }
    return mapper
}

再看看該接口，先建立了一個空的DefaultRESTMapper,而後根據"/api/v1"的groupVersion,遍歷Scheme中全部的kinds，接着再調用mapper.Add(gvk, scope)去填充這個mapper，最後返回該mapper。
看下mapper.Add()的實現：

func (m *DefaultRESTMapper) Add(kind unversioned.GroupVersionKind, scope RESTScope) {
    // resource還分爲單數和複數
    plural, singular := KindToResource(kind)

    // 單數，複數相互轉換
    m.singularToPlural[singular] = plural
    m.pluralToSingular[plural] = singular
    // 根據單複數的resource找到對應的kind
    m.resourceToKind[singular] = kind
    m.resourceToKind[plural] = kind
    // 根據kind找到對應的單複數resource
    m.kindToPluralResource[kind] = plural
    // kind到scope的轉換
    m.kindToScope[kind] = scope
}

RESTMapper其實包含的是一種轉換關係，resource到kind，kind到resource，kind到scope的轉換。resource還分單數和複數。
kind和resource有什麼區別呢？兩者都是字符串，kind是經過Kind=reflector.TypeOf(&Pod{}).Elem().Name()進行取值，去的就是Pod這個結構體的名字。resource是經過plural, singular := KindToResource(kind)取值。singular是將Kind轉換爲小寫字母，而plural是變爲複數。
示例：以Pod爲例，Kind是{Group:"", Version: "v1", Kind: "Pod"},那麼singular是{Group:"", Version: "v1", Kind: "pod"},plural則是{Group："", Version："v1", Resource:"pods"}。
resource要區分單複數，是爲了獲取Pods信息。好比能夠kubectl get pod，也能夠kubectl get pods.

到這裏RESTMapper也基本初始化完了，綜合上面全部的初始化能夠看到，其實主要用internal version和external versions填充Scheme，用external versions去填充GroupMeta以及其成員RESTMapper。
GroupMeta有啥做用呢？主要用於初始化APIGroupVersion。

API資源註冊爲restful api

以前全部的初始化都是爲了這步作鋪墊，上面還有一個APIGroupInfo和APIGroupVersion都沒有進行介紹，這一節都會出現。
當API資源初始化完成之後，須要將這些API資源註冊爲restful api，用來接收用戶的請求。
kube-apiServer使用了go-restful這套框架，裏面主要包括三種對象：

• Container: 一個Container包含多個WebService

• WebService: 一個WebService包含多條route

• Route: 一條route包含一個method(GET、POST、DELETE等)，一條具體的path(URL)以及一個響應的handler function。

API註冊的入口函數有兩個： m.InstallAPIs 和 m.InstallLegacyAPI。
文件路徑：pkg/master/master.go
這兩個函數分別用於註冊"/api"和"/apis"的API,這裏先拿InstallLegacyAPI進行介紹。
這些接口都是在config.Complete().New()函數中被調用：

restOptionsFactory := restOptionsFactory{
        deleteCollectionWorkers: c.DeleteCollectionWorkers,
        enableGarbageCollection: c.GenericConfig.EnableGarbageCollection,
        storageFactory:          c.StorageFactory,
    }
    // 判斷是否使能了用於Watch的Cache
    // 有無cache賦值的是不一樣的接口實現
    // restOptionsFactory.storageDecorator：是一個各個資源的REST interface(CRUD)裝飾者
    // 後面調用NewStorage()時會用到該接口，並輸出對應的CRUD接口及銷燬接口。
    // 能夠參考pkg/registry/core/pod/etcd/etcd.go中的NewStorage()
    // 其實這裏有無cache的接口差別就在於：有cache的話，就提供操做cache的接口；無cache的話，就提供直接操做etcd的接口
    if c.EnableWatchCache {
        restOptionsFactory.storageDecorator = registry.StorageWithCacher
    } else {
        restOptionsFactory.storageDecorator = generic.UndecoratedStorage
    }

    // 判斷/api/v1的group是否已經註冊並enable，是的話再進行install
    if c.GenericConfig.APIResourceConfigSource.AnyResourcesForVersionEnabled(apiv1.SchemeGroupVersion) {
        // 該對象主要提供了一個NewLegacyRESTStorage()的接口
        legacyRESTStorageProvider := corerest.LegacyRESTStorageProvider{
            StorageFactory:       c.StorageFactory,
            ProxyTransport:       c.ProxyTransport,
            KubeletClientConfig:  c.KubeletClientConfig,
            EventTTL:             c.EventTTL,
            ServiceIPRange:       c.ServiceIPRange,
            ServiceNodePortRange: c.ServiceNodePortRange,
            LoopbackClientConfig: c.GenericConfig.LoopbackClientConfig,
        }
        // 進行"/api/v1"的API安裝
        m.InstallLegacyAPI(c.Config, restOptionsFactory.NewFor, legacyRESTStorageProvider)
    }

繼續查看m.InstallLegacyAPI()：

func (m *Master) InstallLegacyAPI(c *Config, restOptionsGetter genericapiserver.RESTOptionsGetter, legacyRESTStorageProvider corerest.LegacyRESTStorageProvider) {
    // 該對象前面介紹過了，比較關鍵，須要深刻查看
    // 返回了RESTStorage和apiGroupInfo，都是重量級的成員
    // 這些初始化也就在這個接口中
    legacyRESTStorage, apiGroupInfo, err := legacyRESTStorageProvider.NewLegacyRESTStorage(restOptionsGetter)
    if err != nil {
        glog.Fatalf("Error building core storage: %v", err)
    }
    // 判斷是否enable了controller,默認是true，這裏跟主題關係不大，暫不深刻
    if c.EnableCoreControllers {
        serviceClient := coreclient.NewForConfigOrDie(c.GenericConfig.LoopbackClientConfig)
        bootstrapController := c.NewBootstrapController(legacyRESTStorage, serviceClient)
        if err := m.GenericAPIServer.AddPostStartHook("bootstrap-controller", bootstrapController.PostStartHook); err != nil {
            glog.Fatalf("Error registering PostStartHook %q: %v", "bootstrap-controller", err)
        }
    }
    // install core Group's API
    if err := m.GenericAPIServer.InstallLegacyAPIGroup(genericapiserver.DefaultLegacyAPIPrefix, &apiGroupInfo); err != nil {
        glog.Fatalf("Error in registering group versions: %v", err)
    }
}

先看下建立APIGroupVersion和RESTStorage對象的接口NewLegacyRESTStorage().
路徑：pkg/registry/core/rest/storage_core.go

func (c LegacyRESTStorageProvider) NewLegacyRESTStorage(restOptionsGetter genericapiserver.RESTOptionsGetter) (LegacyRESTStorage, genericapiserver.APIGroupInfo, error) {
    // 初始化建立一個APIGroupVersion
    apiGroupInfo := genericapiserver.APIGroupInfo{
        // 該GroupMeta是從APIRegistrationManager初始化後的結構體獲取
        GroupMeta:                    *registered.GroupOrDie(api.GroupName),
        VersionedResourcesStorageMap: map[string]map[string]rest.Storage{},
        // 這個api.Scheme以前已經介紹過其初始化了
        Scheme:                      api.Scheme,
        ParameterCodec:              api.ParameterCodec,
        NegotiatedSerializer:        api.Codecs,
        SubresourceGroupVersionKind: map[string]unversioned.GroupVersionKind{},
    }
    // 判斷下autoscaling是否已經註冊並使能，是的話加入到apiGroupInfo.SubresourceGroupVersionKind
    // key是該資源的path
    if autoscalingGroupVersion := (unversioned.GroupVersion{Group: "autoscaling", Version: "v1"}); registered.IsEnabledVersion(autoscalingGroupVersion) {
        apiGroupInfo.SubresourceGroupVersionKind["replicationcontrollers/scale"] = autoscalingGroupVersion.WithKind("Scale")
    }

    var podDisruptionClient policyclient.PodDisruptionBudgetsGetter
    if policyGroupVersion := (unversioned.GroupVersion{Group: "policy", Version: "v1beta1"}); registered.IsEnabledVersion(policyGroupVersion) {
        apiGroupInfo.SubresourceGroupVersionKind["pods/eviction"] = policyGroupVersion.WithKind("Eviction")

        var err error
        podDisruptionClient, err = policyclient.NewForConfig(c.LoopbackClientConfig)
        if err != nil {
            return LegacyRESTStorage{}, genericapiserver.APIGroupInfo{}, err
        }
    }
    // 初始化一個LegacyRESTStorage對象
    // 下面會進行各個接口的初始化，會有Node註冊，IP申請，NodePort申請等等
    restStorage := LegacyRESTStorage{}
    // 建立各種Storage
    podTemplateStorage := podtemplateetcd.NewREST(restOptionsGetter(api.Resource("podTemplates")))

    eventStorage := eventetcd.NewREST(restOptionsGetter(api.Resource("events")), uint64(c.EventTTL.Seconds()))
    limitRangeStorage := limitrangeetcd.NewREST(restOptionsGetter(api.Resource("limitRanges")))

    resourceQuotaStorage, resourceQuotaStatusStorage := resourcequotaetcd.NewREST(restOptionsGetter(api.Resource("resourceQuotas")))
    secretStorage := secretetcd.NewREST(restOptionsGetter(api.Resource("secrets")))
    serviceAccountStorage := serviceaccountetcd.NewREST(restOptionsGetter(api.Resource("serviceAccounts")))
    persistentVolumeStorage, persistentVolumeStatusStorage := pvetcd.NewREST(restOptionsGetter(api.Resource("persistentVolumes")))
    persistentVolumeClaimStorage, persistentVolumeClaimStatusStorage := pvcetcd.NewREST(restOptionsGetter(api.Resource("persistentVolumeClaims")))
    configMapStorage := configmapetcd.NewREST(restOptionsGetter(api.Resource("configMaps")))

    namespaceStorage, namespaceStatusStorage, namespaceFinalizeStorage := namespaceetcd.NewREST(restOptionsGetter(api.Resource("namespaces")))
    restStorage.NamespaceRegistry = namespace.NewRegistry(namespaceStorage)

    endpointsStorage := endpointsetcd.NewREST(restOptionsGetter(api.Resource("endpoints")))
    restStorage.EndpointRegistry = endpoint.NewRegistry(endpointsStorage)

    nodeStorage, err := nodeetcd.NewStorage(restOptionsGetter(api.Resource("nodes")), c.KubeletClientConfig, c.ProxyTransport)
    if err != nil {
        return LegacyRESTStorage{}, genericapiserver.APIGroupInfo{}, err
    }
    restStorage.NodeRegistry = node.NewRegistry(nodeStorage.Node)

    // 建立PodStorage
    // api.Resource("pods")是合成了一個GroupResource的結構
    podStorage := podetcd.NewStorage(
        restOptionsGetter(api.Resource("pods")),
        nodeStorage.KubeletConnectionInfo,
        c.ProxyTransport,
        podDisruptionClient,
    )

    serviceRESTStorage, serviceStatusStorage := serviceetcd.NewREST(restOptionsGetter(api.Resource("services")))
    restStorage.ServiceRegistry = service.NewRegistry(serviceRESTStorage)

    var serviceClusterIPRegistry rangeallocation.RangeRegistry
    serviceClusterIPRange := c.ServiceIPRange
    if serviceClusterIPRange.IP == nil {
        return LegacyRESTStorage{}, genericapiserver.APIGroupInfo{}, fmt.Errorf("service clusterIPRange is missing")
    }

    serviceStorageConfig, err := c.StorageFactory.NewConfig(api.Resource("services"))
    if err != nil {
        return LegacyRESTStorage{}, genericapiserver.APIGroupInfo{}, err
    }

    ServiceClusterIPAllocator := ipallocator.NewAllocatorCIDRRange(&serviceClusterIPRange, func(max int, rangeSpec string) allocator.Interface {
        mem := allocator.NewAllocationMap(max, rangeSpec)
        // TODO etcdallocator package to return a storage interface via the storageFactory
        etcd := etcdallocator.NewEtcd(mem, "/ranges/serviceips", api.Resource("serviceipallocations"), serviceStorageConfig)
        serviceClusterIPRegistry = etcd
        return etcd
    })
    restStorage.ServiceClusterIPAllocator = serviceClusterIPRegistry

    var serviceNodePortRegistry rangeallocation.RangeRegistry
    ServiceNodePortAllocator := portallocator.NewPortAllocatorCustom(c.ServiceNodePortRange, func(max int, rangeSpec string) allocator.Interface {
        mem := allocator.NewAllocationMap(max, rangeSpec)
        // TODO etcdallocator package to return a storage interface via the storageFactory
        etcd := etcdallocator.NewEtcd(mem, "/ranges/servicenodeports", api.Resource("servicenodeportallocations"), serviceStorageConfig)
        serviceNodePortRegistry = etcd
        return etcd
    })
    restStorage.ServiceNodePortAllocator = serviceNodePortRegistry

    controllerStorage := controlleretcd.NewStorage(restOptionsGetter(api.Resource("replicationControllers")))

    serviceRest := service.NewStorage(restStorage.ServiceRegistry, restStorage.EndpointRegistry, ServiceClusterIPAllocator, ServiceNodePortAllocator, c.ProxyTransport)

    // 初始化了一個restStorage的map，而後賦值給APIGroupInfo.VersionedResourcesStorageMap["v1"]
    restStorageMap := map[string]rest.Storage{
        "pods":             podStorage.Pod,
        "pods/attach":      podStorage.Attach,
        "pods/status":      podStorage.Status,
        "pods/log":         podStorage.Log,
        "pods/exec":        podStorage.Exec,
        "pods/portforward": podStorage.PortForward,
        "pods/proxy":       podStorage.Proxy,
        "pods/binding":     podStorage.Binding,
        "bindings":         podStorage.Binding,

        "podTemplates": podTemplateStorage,

        "replicationControllers":        controllerStorage.Controller,
        "replicationControllers/status": controllerStorage.Status,

        "services":        serviceRest.Service,
        "services/proxy":  serviceRest.Proxy,
        "services/status": serviceStatusStorage,

        "endpoints": endpointsStorage,

        "nodes":        nodeStorage.Node,
        "nodes/status": nodeStorage.Status,
        "nodes/proxy":  nodeStorage.Proxy,

        "events": eventStorage,

        "limitRanges":                   limitRangeStorage,
        "resourceQuotas":                resourceQuotaStorage,
        "resourceQuotas/status":         resourceQuotaStatusStorage,
        "namespaces":                    namespaceStorage,
        "namespaces/status":             namespaceStatusStorage,
        "namespaces/finalize":           namespaceFinalizeStorage,
        "secrets":                       secretStorage,
        "serviceAccounts":               serviceAccountStorage,
        "persistentVolumes":             persistentVolumeStorage,
        "persistentVolumes/status":      persistentVolumeStatusStorage,
        "persistentVolumeClaims":        persistentVolumeClaimStorage,
        "persistentVolumeClaims/status": persistentVolumeClaimStatusStorage,
        "configMaps":                    configMapStorage,

        "componentStatuses": componentstatus.NewStorage(componentStatusStorage{c.StorageFactory}.serversToValidate),
    }
    if registered.IsEnabledVersion(unversioned.GroupVersion{Group: "autoscaling", Version: "v1"}) {
        restStorageMap["replicationControllers/scale"] = controllerStorage.Scale
    }
    if registered.IsEnabledVersion(unversioned.GroupVersion{Group: "policy", Version: "v1beta1"}) {
        restStorageMap["pods/eviction"] = podStorage.Eviction
    }
    // 將上面的restStorageMap賦值給v1
    apiGroupInfo.VersionedResourcesStorageMap["v1"] = restStorageMap

    return restStorage, apiGroupInfo, nil
}

看完這個接口後，咱們繼續回到前面，看下m.GenericAPIServer.InstallLegacyAPIGroup()接口：
路徑：pkg/genericapiserver/genericapiserver.go

func (s *GenericAPIServer) InstallLegacyAPIGroup(apiPrefix string, apiGroupInfo *APIGroupInfo) error {
    // 判斷前綴參數是否正確
    if !s.legacyAPIGroupPrefixes.Has(apiPrefix) {
        return fmt.Errorf("%q is not in the allowed legacy API prefixes: %v", apiPrefix, s.legacyAPIGroupPrefixes.List())
    }
    // 關鍵接口，真正install API
    if err := s.installAPIResources(apiPrefix, apiGroupInfo); err != nil {
        return err
    }

    // 獲取了該Group下全部的version信息
    // 應該用於發現當前的全部版本信息
    apiVersions := []string{}
    for _, groupVersion := range apiGroupInfo.GroupMeta.GroupVersions {
        apiVersions = append(apiVersions, groupVersion.Version)
    }
    // Install the version handler.
    // Add a handler at /<apiPrefix> to enumerate the supported api versions.
    apiserver.AddApiWebService(s.Serializer, s.HandlerContainer.Container, apiPrefix, func(req *restful.Request) *unversioned.APIVersions {
        clientIP := utilnet.GetClientIP(req.Request)

        apiVersionsForDiscovery := unversioned.APIVersions{
            ServerAddressByClientCIDRs: s.discoveryAddresses.ServerAddressByClientCIDRs(clientIP),
            Versions:                   apiVersions,
        }
        return &apiVersionsForDiscovery
    })
    return nil
}

那咱們繼續進入關鍵接口s.installAPIResources(apiPrefix, apiGroupInfo):

func (s *GenericAPIServer) installAPIResources(apiPrefix string, apiGroupInfo *APIGroupInfo) error {
    // 遍歷該Group下的全部GroupVersons
    for _, groupVersion := range apiGroupInfo.GroupMeta.GroupVersions {
        // 建立APIGroupVersion
        apiGroupVersion, err := s.getAPIGroupVersion(apiGroupInfo, groupVersion, apiPrefix)
        if err != nil {
            return err
        }
        if apiGroupInfo.OptionsExternalVersion != nil {
            apiGroupVersion.OptionsExternalVersion = apiGroupInfo.OptionsExternalVersion
        }
        // 根據以前建立的APIGroupVersion,而後安裝restful API
        // 該s.HandlerContainer.Container就是go-restful的Container
        if err := apiGroupVersion.InstallREST(s.HandlerContainer.Container); err != nil {
            return fmt.Errorf("Unable to setup API %v: %v", apiGroupInfo, err)
        }
    }

    return nil
}

func (s *GenericAPIServer) getAPIGroupVersion(apiGroupInfo *APIGroupInfo, groupVersion unversioned.GroupVersion, apiPrefix string) (*apiserver.APIGroupVersion, error) {
    storage := make(map[string]rest.Storage)
    // 若是是核心組的話，Version爲"v1",該VersionedResourcesStorageMap的初始化要看
    // 以前的NewLegacyRESTStorage()接口，在該接口中進行的初始化
    // 遍歷全部的ResourcesStorage，並賦值給storage
    for k, v := range apiGroupInfo.VersionedResourcesStorageMap[groupVersion.Version] {
        storage[strings.ToLower(k)] = v
    }
    // 建立APIGroupVersion
    version, err := s.newAPIGroupVersion(apiGroupInfo, groupVersion)
    // 設置Prefix, 核心組的話是"/api"
    version.Root = apiPrefix
    version.Storage = storage
    return version, err
}

到這裏從API資源到restful API,就已經註冊完成了。
至於apiGroupVersion.InstallREST()接口，咱們這裏先簡單介紹，後面會另起一篇文章結合go-restful進行介紹。
InstallREST()接口路徑：pkg/apiserver/apiserver.go

func (g *APIGroupVersion) InstallREST(container *restful.Container) error {
    installer := g.newInstaller()
    ws := installer.NewWebService()
    apiResources, registrationErrors := installer.Install(ws)
    lister := g.ResourceLister
    if lister == nil {
        lister = staticLister{apiResources}
    }
    AddSupportedResourcesWebService(g.Serializer, ws, g.GroupVersion, lister)
    container.Add(ws)
    return utilerrors.NewAggregate(registrationErrors)
}

func (a *APIInstaller) Install(ws *restful.WebService) (apiResources []unversioned.APIResource, errors []error) {
    errors = make([]error, 0)

    proxyHandler := (&ProxyHandler{
        prefix:     a.prefix + "/proxy/",
        storage:    a.group.Storage,
        serializer: a.group.Serializer,
        mapper:     a.group.Context,
    })

    // Register the paths in a deterministic (sorted) order to get a deterministic swagger spec.
    paths := make([]string, len(a.group.Storage))
    var i int = 0
    for path := range a.group.Storage {
        paths[i] = path
        i++
    }
    sort.Strings(paths)
    for _, path := range paths {
        // 該接口是關鍵，最終將一個rest.Storage對象轉換成實際的restful api，好比getter、lister等處理函數，並將實際的URL關聯起來
        apiResource, err := a.registerResourceHandlers(path, a.group.Storage[path], ws, proxyHandler)
        if err != nil {
            errors = append(errors, fmt.Errorf("error in registering resource: %s, %v", path, err))
        }
        if apiResource != nil {
            apiResources = append(apiResources, *apiResource)
        }
    }
    return apiResources, errors
}

在這個註冊的過程當中，InstallREST最終調用了registerResourceHandlers()接口，該接口最終會把一個rest.Storage對象轉換成實際的getter、lister等處理函數，並和實際的URL關聯起來。

用戶參數配置

• runtime-config： 用於enable/disable extensions group。默認的狀況下DaemonSets、Deployments、HorizontalPodAutoscalers、Ingress、Jobs和ReplicaSets是使能的,還有v1下的默認都是使能的。另外的功能就能夠經過該配置進行設置. 例如：disable deployments： --runtime-config=extensions/v1beta1/deployments=false.

參考資料

1.api-group.md: https://github.com/kubernetes...