XAPI Dispatch

XAPI Service Dispatch and Message Forwarding

This page partly describes the service dispatch and message forwarding mechanism in xapi. The authoritative source of wisdom remains the source code.

Incoming Call Dispatch

In xapi/xapi.ml, HTTP servers are run on both a Unix domain socket and on an Internet port. An HTTP handler is added for incoming HTTP POST requests at URI "/":

Xapi_http.add_handler Http.Post "/" (Api_server.callback false)

The HTTP handler will thus invoke Api_server.callback for incoming XML-RPC requests. This function, in xapi/api_server.ml, parses the body.

If the slave is in emergency mode and the call is not an emergency call, an error is raised. If this instance of xapi is not the pool master, and the call came in via the Unix domain socket, and the call is not a "login" call, then forward the request to the master using the "forward" function. Otherwise, Server.dispatch_xml is invoked:

let response = Server.dispatch_xml req fd xml

The dispatch_xml function (defined in autogen/server.ml) starts a new task to handle the call and compares the incoming call against a long list of potential calls. When a match is found, Server_helpers.do_dispatch is called, for example:

Server_helpers.do_dispatch ~session_id ~forward_op "<none/>" __async supports_async

  __call local_op marshaller fd http_req __label generate_task_for

The parameters to do_dispatch indicate a variety of things such as whether the call can be performed asynchronously, how the arguments are to be unmarshalled, etc. Also provided are a "local_op" and a "forward_op" which describe what to do if the call is to be serviced locally or forwarded, respectively.

The do_dispatch function (defined in idl/ocaml_backend/server_helpers.ml) creates a new thread for servicing the call for those which are executed asynchronously. It then calls "exec_with_context" which in turn calls "exec" which executes the local_op if it's not the master or the forward_op if it is the master.

The local_op specified in autogen/server.ml sometimes delves straight into the datamodel (e.g. via Db_actions.DB_Action, defined in autogen/db_actions.ml) but often refer to a function in Custom. This module is passed in via the Make functor and is instantiated with the Actions module from xapi/api_server.ml which merely translates sub-module names such as Host to the Xapi_host module whose implementation can be found in xapi/xapi_host.ml. Hence a mention of Custom.Host.disable_binary_storage refers to the implementation of the function "disable_binary_storage" in xapi/xapi_host.ml. These functions describe the logic requested by the call.

The forward_op often makes reference to a function in Forward, which again is a module passed in via the Make functor, instantiated with the result of applying the Actions module from xapi/api_server.ml to the Message_forwarding.Forward functor, defined in xapi/message_forwarding.ml. Hence a mention of Forward.Host.disable_binary_storage refers to the implementation of the function "disable_binary_storage" in the Forward module in xapi/message_forwarding.ml, described in the next section.

Message Forwarding

In the case where Server_helpers.do_dispatch elects to execute the forward_op rather than the local_op, the request must be forwarded to the appropriate host, which may even be the localhost. This is dealt with by the message forwarding layer, implemented in xapi/message_forwarding.ml.

The Forward functor contains a number of modules, such as Pool, Host, etc. Each module contains functions which take a context, a host, plus any arguments corresponding to the parameters of the API call. Typically, these functions invoke "do_op_on", passing both a "local_fn" and an "op":

let do_op_on ~local_fn ~__context ~host op = ...

The local_fn is usually a function from the Local module. This module is passed in via the Forward functor and is instantiated with the Actions module from xapi/api_server.ml (see above). Hence a mention of Local.Host.disable_binary_storage refers to the implementation of the function "disable_binary_storage" in xapi/xapi_host.ml.

The op is a function which performs an RPC to the appropriate host.

The do_op_on function checks that the target host is alive, otherwise raising a "host offline" server error then defers to the do_op_on_common function. This function checks whether the target host is actually the local host. If so, then the local_fn is executed. Otherwise, an expression such as the following is evaluated:

call_slave_with_session remote_rpc_retry __context host

  (set_forwarding_on_task context host)

  (fun session_id rpc -> Client.Host.disable_binary_storage rpc session_id host)

The function call_slave_with_session first logs in to the host, getting a session_id in return:

let session_id = Xapi_session.login_no_password ~__context ~uname:None ~host ~pool:true

It then determines the address of the host by querying the local database:

let hostname = Db.Host.get_address ~__context ~self:host

It then invokes the function passed in as the final argument, passing it the session_id and a function which performs the RPC to the remote host. An expression like the following is evaluated:

Client.Host.disable_binary_storage

  (remote_rpc_retry context hostname (set_forwarding_on_task context host))

  session_id host

The sub-expression "set_forwarding_on_task context host" finds a task relating to the given context in the database, and sets "forwarded" to true and "forwarded_to" to the host on it, returning the task if found.

The sub-expression "remote_rpc_retry context hostname task_opt" is a function which takes an XML string (from the Xml.xml type) and performs the RPC using the XML-RPC library in idl/ocaml_backend/xmlrpcclient.ml.

Hence the expression above boils down to:

Client.Host.disable_binary_storage (function xml -> [engp:send the XML-RPC request to the host]) session_id host

The implementation of the Client module can be found in autogen/client.ml. This uses the rpc_wrapper function to perform the RPC, generating the XML for the call with its arguments and parsing the return value.