 ߷׻ؿ

 giant_lock ݻϵ̲ƤϤʤXXX ײ
I/O ԤʤäƤϤʤ
ͣ㳰ϡdbq Ԥε̲ϻʤ

ȴǡͥåȥgiant_lock ݻޤޤäƤ
Ȥ롣Ϥ nonblocking ⡼ɤǹԤʤäƤ(Ȧ?)ʤΤǡ
ƤХåե⾮С̲ʤȦ
/* XXX FIXME too long giant lock */ ȤȤǡ
ƤβսޡƤȦ

 ᥿ǡ򡢥˥å夹롣
ǽͥܥǥ󥰤ñˤΤŪ
ǡ gfmdưƥɤ߹ߡΤӤ dbq ͳ DB 
񤭽Фȡư DB ɤ߹ߤԤʤȤϤʤ

xattr 㳰
ͳϡ
xattr ȤƤʤ礭ʥǡꤷƤ
߷׼Ԥ¾ʬȤ
Ǥ뤿ᡣ
դˡǡȤƾ xattr ؤΥå夵Ƥ餺
٤Ȥ»ब롣

 mutex 

mutex ֤ΰ¸ط˥롼פäƤϤʤǥåɥå롣ס

Ʊʣ mutex γԤʤγϡ٤ʸ񲽤ɬס

mutex ϡserver/gfmd/README 
XXX ݼ餵Ƥʤ׹

giant_lock -> struct host::replication_mutex -> dbq.mutex
	cf. host_peer_set()

giant_lock -> dfc_allq.mutex
giant_lock -> busyq.mutex -> removeq.mutex
	ʲξʣˤ롣
	giant_lock -> removeq.mutex
	busyq.mutex -> removeq.mutex

 giant_lock ݻޤޡ̲ɤå

  ʲΥåɤΤߡgiant_lock ݻޤ޵̲Ƥɤ
  ʳΥåɤ̲Τϡ¸طΥ롼פ뤿
  ػߡ

  sync_protocol_thread_pool °륹å

 leaf routine ΤߤǳƤ mutex

    mutex ݻޤޡ¾ mutex 뤳ȤϤʤ
   ƤϤʤ

  callout_module.mutex

 åɰ

åɴ֤ΰ¸ط˥롼פäƤϤʤǥåɥå롣ס

thrpool_add_job() 硢
thrpool_add_job() ƤӽФåɤϡ
thrpool_add_job() 륹åɥסˡ
¸Ƥޤ
ʤthrpool_add_job() 륹åɥס뤬դξ硢
thrpool_add_job() 륹åɤϵ̲롣

åɥס°륹åɤ
ȤΥåɥסФ thrpool_add_job() Τܡס
Υåɥסåɤ
ޤƱ˼åɥסФ thrpool_add_job() ȡ
ס˶ʤΤǡƵ̲Τޤ޿ˤʤ롣
äȤ̾٤ǤСgfarm_metadb_job_queue_length Ĺ
塼ˤäƴ˾פ뤿ᡢΥǥåɥåȯʤ٤
塼դλ˾嵭ξȯȤޤ

åɥס֤ΰ¸ط˥롼פΤܡס
back_channel_send_thread_pool ϡ̿򤷤ưŤ, 
back_channel_recv_thread_pool ˰¸ƤΤաס

¸طοޤ
doc/internal/png/gfmd-thread-dependency.png
ˤ뤬οޤǡ
(å)éäåɴ֤롼פˤʤäƤƤϤʤ
Ȣ(åɥס)éäס֤롼פˤʤäƤƤϤʤ
ƱȢ(ƱΥåɥס)ˤ륹åɴ֤ˡ
  thrpool_add_job()طäƤϤʤ

 main å: accepting_loop() 

   TCP ³ԤԤʤΤ䡣

 create_detached_thread() Ǻ롢Ωåɡ

ʥƻ sigs_handler()

callout ƻ callout_main()  CALLOUT_NTHREADS

    callout_main()  thrpool_add_job() 륹åɥסο
    callout_main() åɤѰդɤʤȡ륹å
    ס뤬դȤʤäȤطʤåɤνޤٱ䤷Ƥޤ
    ߤϡback_channel_send_thread_pool ΤߤǤ뤿ᡢCALLOUT_NTHREADS == 1

    callout_reset() ƤΤϡback_channel_recv_thread_pool °
    gfs_async_client_status_result() callout_reset() 
    cond_signal() ƤʤΤǡcallout_reset() Ǥϰ¸ϵʤ

  db_thread()

ͥåȥƻ peer_watcher()  1

    peer_watcher()  thrpool_add_job() 륹åɥסο
    peer_watcher() åɤѰդɤʤȡ륹å
    ס뤬դȤʤäȤطʤåɤνޤٱ䤷Ƥޤ
    ߡsync_protocol_thread_pool  back_channel_recv_thread_pool  2
    뤿 peer_watcher() åɤ 2ĤˤԤ peer_watcher() 
    Ƥʤʤᡢ1ĤΤޤޡ
	 peer_watcher()  2åɤư褦ѹͽꡣXXX

    XXX DEADLOCK
    ʲͳǡdeadlock δ롣

    peer_watcher() ƤӽФ 2ĤΥåɥס롢sync_protocol_thread_pool
     back_channel_recv_thread_pool ϡback_channel_send_thread_pool 
    褦ˡƱ̤٤˹ԤʤäƵͤޤ뿴ۤϤʤ
    sync_protocol_thread_pool  back_channel_send_thread_pool ˰¸
    Ƥ뤿ᡢback_channel_send_thread_pool ¦ǡpeer_watcher() 
     sync_protocol_thread_pool Ф thrpool_add_job() ͤޤ
    ǽϻĤ롣
    peer_watcher()  1åɤΤߤȡpeer_watcher() Τߤޤä
    ޤᡢback_channel_recv_thread_pool ⴬ź򤯤äơߤޤ롣
    η̡back_channel_recv_thread_pool ˰¸Ƥ
    back_channel_send_thread_pool ưʤʤꡢ¸طΥ롼פȤʤäơ
    deadlock 롣

  peer_closer()

  backend_protocolξ硢filesystem nodeФ remover()  

    ѻߤơback_channel_recv_thread_pool Ȥ褦ˤͽꡣ

 åɥס

סγƥåɤμΤϡthrpool_worker()

 authentication_thread_pool

    ΥסѤƵưΤϡʲΥåɡ

    try_auth()
      accepting_loop()  thrpool_add_job() 롣

 sync_protocol_thread_pool

    ΥסѤƵưΤϡʲΥåɡ

    protocol_main()
      - try_auth() peer_authorized() ͳǡthrpool_add_job() 롣
      - peer_watcher() thrpool_add_job() 롣

 back_channel_send_thread_pool
  
    ΥסѤƵưΤϡʲΥåɡ

    gfs_async_client_status_request()
      - protocol_main() gfm_server_switch_async_back_channel() ͳǡ
        thrpool_add_job() 롣
      - callout_main() thrpool_add_job() 롣
	callout ˵ꤷƤΤϡback_channel_recv_thread_pool °
	gfs_async_client_status_result() callout_reset() ȤäƤΤ
	¸ϵʤ

    gfs_async_client_replication_request_request()
      - protocol_main() async_back_channel_replication_request() ͳǡ
        thrpool_add_job() 롣

 back_channel_recv_thread_pool

    ΥסѤƵưΤϡʲΥåɡ

    async_back_channel_main()
      - protocol_main() gfm_server_switch_async_back_channel() ͳǡ
        thrpool_add_job() 롣
      - peer_watcher() thrpool_add_job() 롣
	줬ȯΤϡgfsd  async RPC request 뤤ϡ
	gfmd  async RPC request Ф gfsd  reply 硣
	Ԥ gfmdgfsd  async RPC request ϡback_channel_send_thread_pool
	ѤƹԤʤ롣Τᡢasync_back_channel_main() ν
	ͤޤȡ̤Ȥơback_channel_send_thread_pool ޤǤͤޤ
	ǽ롣
	Τᡢback_channel_recv_thread_pool °륹åɤϡ
	ʲΥס°񸻤ԤäƤϤʤס
	 * authentication_thread_pool
	 * sync_protocol_thread_pool
	 * back_channel_send_thread_pool
	⤷Ԥʤȡ¸ط롼פdeadlock δ롣
	嵭Υס thrpool_add_job() Ƥ⤤ʤ
	ޤ嵭Υס°Ԥ mutex ԤäƤ⤤ʤ
	äơgiant_lock() ԤʤäƤܡ
	嵭Υסȶ礹뤬켫ȤǤ cond_wait() 
	Ԥʤʤ꡼դ mutex ԤĤʤʤ
	ޤgfm_async_server_replication_result() ϡľܡʲ
	ԤʤäƤϤʤ
		̤νΤ host_replicated()
		̤ peer_sender_lock() ԤĤȤ롣
				 ¦ԤĤȤ롣XXX DEADLOCK

 back_channel ˴ؤ߷

 back_channel ɬפʻ (callout ʤ) 
   (1) peer ˵°뤫
   (2) host ˵°뤫
   (3) back_channel ̿Τι¤Τ˵°뤫
  Ȥ褬롣
  back_channel ̿λƱͤȡ¸֤ gfmd ư
  Ǥ (1)  (2) ǡ¤ΤäƤޤƱѤΰ
  γݤɬפȤʤ (3) ץκ꤬ñˤʤ롣
  ޤʲΤ褦 peer host ¿
	(peer ο == 饤Ȥο 2  host ο)
  Ūˤ (2) ͭʤΤǡhost ˵°뤳Ȥˤ
   async ˤĤƤϡconn Ʊͤ̿ɬפʥǡǤꡢ
  back_channel ͥڤؤäƤ뤬
  ŤͥޤĤäƤ˻Ȥǽθ (1) Ȥ롣

host_receiver_lock() ɬפͳ
   peer_watch() ƤӽФ졢Ʊ peer ʤƱ쥹åɤư
  뤿ᡢpeer ֤μȯʤ
  host_receiver_lock() Ƥͳϡpeer_free_request() Ȥζ
  Τᡣ

host_sender_lock()/_trylock() ɬפͳ
  - host_receiver_lock() Ʊ peer_free_request() Ȥζ
  - ξƱ peer ǤʣåɤǶ礹ΤǤβ
  - ¦ͤޤäˡƱ peer Υåɤǡåɥס뤬
    ԤΤɤ host_sender_trylock() ǻ롣
    gfm_async_server_replication_result() Τߤ _trylock() ǤϤʤ
    _lock() ȤäƤΤǡʣΥåɥסͭƤޤ
    ǽ뤬Ǥ뤿ᡢ¼ŪʴϤʤ㤤

ۥ down к
  - gfs_client_status_request() ǡ󤫤ޤʤä
    硢ʤ host_status_reply_is_waiting() ΩäƤ
    ˤϡͥǤ롣(down ֤Ȥʤ)
  - λ host_sender_try_lock() ǽɤ
    ǧαƤˤϡhost_peer_busy() 
    ͥǤ

 XXX ̤ TO-DO

protocol_main() Ǽ resuming_thread() ǤԤˤʤ
  Τ뤬resume  callback Ǽ¸Ƥ뤿ᡢåɥס
  ԤΥǥåɥåˤϤʤʤ

