rgw object read and write

1、Get Object。

一、讀取Object的主要處理流程。

RGWGetObj::execute()

|__建立RGWGetObj_CB類實例，其中handle_data()函數爲回調函數，該函數會調用RGWGetObj::get_data_cb()函數，而該函數最終會調用send_reponse_data()

|__建立RGWRados::Object類實例

|__建立RGWRados::Object::Read類實例

|__get_params() 從HTTP請求字符串中讀取range_str/if_mod/if_unmod/if_match/if_nomatch信息

|__init_common() 解析range_str/if_mod/if_unmod內容

|__RGWRados::Object::Read::prepare()

    |__RGWRados::get_state()

|__RGWRados::get_obj_state()

    |__RGWRados::get_obj_state_impl()

|__RGWObjectCtx::get_state()

|__RGWRados::raw_obj_stat()

|__檢查attrset中是否包含RGW_ATTR_SHADOW_OBJ屬性並將屬性值寫入到RGWObjState::shadow_obj中

|__檢查attrset中是否包含RGW_ATTR_MANIFEST屬性並將屬性值寫入到RGWObjState::manifest中，同時更新has_manifest=true以及size值

|__檢查attrset中是否包含RGW_ATTR_PG_VER屬性並將屬性值寫入到RGWObjState::pg_ver中

|__檢查attrset中是否包含RGW_ATTR_SOURCE_ZONE屬性並將屬性值寫入到RGWObjState::zone_short_id中

|__檢查attrset中是否包含RGW_ATTR_OLH_ID_TAG屬性並將屬性值寫入到RGWObjState::olh_tag中

    |__RGWRados::get_obj_ioctx()

|__get_obj_bucket_and_oid_loc()

|__open_bucket_data_ctx()

    |__檢查屬性值中關於mod_ptr/unmod_ptr/if_match/if_unmatch是否有效

|__RGWGetObj::handle_user_manifest()

    |__解析prefix字符串且以」/」爲分隔符，解析出bucket_name和obj_prefix

    |__獲得rgw_bucket/RGWBucketInfo/RGWAccessControlPolicy實例

    |__iterate_user_manifest_parts() 找到object的total_len以及object已經完成的字節數

    |__iterate_user_manifest_parts()

|__get_obj_user_manifest_iterate_cb()

    |__RGWGetOp::read_user_manifest_part()

|__建立RGWRados::Object類實例

|__建立RGWRados::Object::Read類實例

|__RGWRados::Object::Read::prepare()

|__rgw_policy_from_attrsets() 從attrs屬性列表中找到RGW_ATTR_ACL屬性列表值

|__verify_object_permission() 校驗bucket ACL和object ACL的合法性

|__RGWRados::Object::Read::read()

|__send_response_data()

|__RGWGetObj::handle_slo_manifest()

    |__iterate_slo_parts()

|__get_obj_user_manifest_iterate_cb()

    |__RGWGetObj::read_user_manifest_part()

|__建立RGWRados::Object類實例

|__建立RGWRados::Object::Read類實例

|__RGWRados::Object::Read::prepare()

|__rgw_policy_from_attrsets() 從attrs屬性列表中找到RGW_ATTR_ACL屬性列表值

|__verify_object_permission() 校驗bucket ACL和object ACL的合法性

|__RGWRados::Object::Read::read()

|__send_response_data()

|__RGWRados::Object::Read::iterate()

    |__構建struct get_obj_data實例

    |__RGWRados::iterate_obj()

|__RGWRados::get_obj_state()

|___get_obj_iterate_cb()

    |__RGWRados::get_obj_iterate_cb()

|__建立ObjectReadOperation類實例

|__ObjectReadOperation::read()

|__librados::IoCtx::aio_operate() 執行讀取對象內容操做

|__RGWRados::flush_read_list()

    |__RGWGetObj_CB::handle_data()

     |__RGWGetObj::get_data_obj()

    |__RGWGetObj_ObjStore_S3::send_response_data()

|__send_response_data()

2、Put Object。

一、上傳Object的主要處理流程(非分片)。

RGWPutObj::execute()

|__RGWPutObj_ObjStore::get_params() 從HTTP請求中讀取HTTP_CONTENT_MD5值

|__get_system_versioning_params() 從HTTP請求中讀取versioned-epoch和version-id值

|__ceph_unarmor() 計算supplied_md5_bin數組的值

|__RGWRados::check_quota() 獲取user和bucket的quota信息

|__RGWPutObj::select_processor() 獲得RGWPutObjProcessor_Atomic類實例

|__RGWPutObjProcessor_Atomic::prepare()

    |__RGWPutObjProcessor_Atomic::prepare_init()

|__RGWRados::get_max_chunk_size() 獲得max_chunk_size值

    |__RGWObjManifest::set_trivial_rule()

    |__RGWObjManifest::generator::create_begin()

|__RGWObjManifest::get_rule()

|__RGWObjManifest::get_head_size()

|__RGWObjManifest::get_implicit_location()

對於第一次上傳且所上傳的數據小於max_head_size，則不分片；

對於第一次上傳但所上傳的數據大於max_head_size，則oid要加上cur_stripe信息且rgw_obj的namespace是」shadow」；

對於非第一次上傳但上傳整個分片數據，則oid要加上.cur_part_id信息且rgw_obj的namespace是」multipart」；

對於非第一次上傳且上傳非整個分片數據，則oid要加上.cur_part_id_cur_stripe信息且rgw_obj的namespace是」shadow」；

|__RGWObjManifest::update_iterators()

|__RGWPutObj_ObjStore_S3::get_data()

    |__RGWPutObj_ObjStore::get_data()

|__STREAM_IO()->read()

|__put_data_and_throttle()

    |__RGWPutObjProcessor_Atomic::handle_data()

|__RGWPutObjProcessor_Atomic::prepare_next_part()

|__RGWPutObjProcessor_Atomic::write_data()

    |__RGWPutObjProcessor_Aio::handle_obj_data()

|__RGWRados::aio_put_obj_data()

    |__RGWPutObjProcessor_Atomic::throttle_data()

|__RGWPutObjProcessor_Aio::throttle_data()

|__RGW_Auth_S3::authorize_aws4_auth_complete()

|__RGWRados::check_quota()

|__RGWPutObjProcessor_Atomic::complete_hash()

|__設置RGW_ATTR_ACL/RGW_ATTR_ETAG屬性

|__rgw_get_requst_metadata() 從HTTP請求中讀取RGW_ATTR_PREFIX屬性

|__encode_delete_at_attr() 設置RGW_ATTR_DELETE_AT屬性

|__RGWPutObjProcessor::complete()

    |__RGWPutObjProcessor_Atomic::do_complete()

|__RGWPutObjProcessor_Atomic::complete_writing_data()

    |__RGWPutObjProcessor_Atomic::write_data()

    |__RGWPutObjProcessor_Aio::throttle_data()

    |__RGWPutObjProcessor_Atomic::complete_parts()

    |__RGWPutObjProcessor_Aio::drain_pending()

|__RGWRados::Object::Write::write_meta() 寫object的metadata，最後更新quota使用量cache

|__dispose_processor()

二、上傳Object的主要處理流程(分片)。

RGWPutObj::execute()

|__RGWPutObj_ObjStore::get_params() 從HTTP請求中讀取HTTP_CONTENT_MD5值

|__get_system_versioning_params() 從HTTP請求中讀取versioned-epoch和version-id值

|__ceph_unarmor() 計算supplied_md5_bin數組的值

|__RGWRados::check_quota() 獲取user和bucket的quota信息

|__RGWPutObj::select_processor() 獲得RGWPutObjProcessor_Atomic類實例

|__RGWPutObjProcessor_Multipart::prepare()

    |__RGWPutObjProcessor_Atomic::prepare_init()

|__RGWRados::get_max_chunk_size() 獲得max_chunk_size值

    |__RGWMPObj::init() 初始化multipart object

    |__RGWObjManifest::set_multipart_part_rule()

    |__RGWObjManifest::generator::create_begin()

|__RGWObjManifest::get_rule()

|__RGWObjManifest::get_head_size()

|__RGWObjManifest::get_implicit_location()

對於第一次上傳且所上傳的數據小於max_head_size，則不分片；

對於第一次上傳但所上傳的數據大於max_head_size，則oid要加上cur_stripe信息且rgw_obj的namespace是」shadow」；

對於非第一次上傳但上傳整個分片數據，則oid要加上.cur_part_id信息且rgw_obj的namespace是」multipart」；

對於非第一次上傳且上傳非整個分片數據，則oid要加上.cur_part_id_cur_stripe信息且rgw_obj的namespace是」shadow」；

|__RGWPutObj_ObjStore_S3::get_data()

    |__RGWPutObj_ObjStore::get_data()

|__STREAM_IO()->read()

|__put_data_and_throttle()

    |__RGWPutObjProcessor_Atomic::handle_data()

|__RGWPutObjProcessor_Atomic::prepare_next_part()

|__RGWPutObjProcessor_Atomic::write_data()

    |__RGWPutObjProcessor_Aio::handle_obj_data()

|__RGWRados::aio_put_obj_data()

    |__RGWPutObjProcessor_Atomic::throttle_data()

|__RGWPutObjProcessor_Aio::throttle_data()

|__RGW_Auth_S3::authorize_aws4_auth_complete()

|__RGWRados::check_quota()

|__RGWPutObjProcessor_Atomic::complete_hash()

|__設置RGW_ATTR_ACL/RGW_ATTR_ETAG屬性

|__rgw_get_requst_metadata() 從HTTP請求中讀取RGW_ATTR_PREFIX屬性

|__encode_delete_at_attr() 設置RGW_ATTR_DELETE_AT屬性

|__RGWPutObjProcessor::complete()

    |__RGWPutObjProcessor_Multipart::do_complete()

|__RGWPutObjProcessor_Multipart::complete_writing_data()

    |__RGWPutObjProcessor_Multipart::write_data()

    |__RGWPutObjProcessor_Aio::throttle_data()

    |__RGWPutObjProcessor_Multipart::complete_parts()

    |__RGWPutObjProcessor_Aio::drain_pending()

|__RGWRados::Object::Write::write_meta() 寫object的metadata，最後更新quota使用量cache

|__建立RGWUploadPartInfo對象且設置初始化該對象

|__初始化multipart的rgw_obj對象且設置namespace以及設置in_extra_data=true

|__RGWRados::omap_set() 設置對象的omap信息且omap的key＝part.part_num

|__dispose_processor()