Connector connector for Enterprise Search Engine Development (6)

Before continuing to analyze the source code, it is necessary to familiarize yourself with the UML Model diagram of the connector. Otherwise, in the face of the intricate dependency, it is inevitable that people cannot clarify their clues.

First, familiarize yourself with the following UML Model:

The illustration I drew is incomplete, so that the UML Model of the connector can be clearly expressed to avoid interference in details.

The connectorcoordinatorimpl class uses the member variable threadpool to call the cancelbatch class that implements the multi-threaded interface to implement the connector collection function.

Multi-threaded cancelbatch calls the traversalmanager interface instance of the connector by calling a traverser-type object (querytraverser) to traverse data.

Others are undoubtedly auxiliary classes. The batchresultrecorder class is used to record the collection result information; The traversalstatestore is used to store the status; and The batchsize is used to pass in the batch size information;

The traversalcontext interface used by feedconnection to publish xmlfeed data like a search engine application is context information (the scheduling class of the traversalmanger interface must be implemented while the traversalcontextaware interface must be implemented)

The actual connector class model is far more complex than above. Is it against the single responsibility principle that individual classes implement the above interfaces at the same time?

Among them, the batchcoordinator class plays a pivot role. This class simultaneously implements three interfaces: traversalstatestore, batchresultrecorder, and batchtimeout. The UML Model diagram of the related interfaces and classes is as follows:

 

 

Therefore, the batchcoordinator class in the program can adapt to different types, similar to the role of multiple people.

Next, let's review the header to analyze the startbatch () method of the connectorcoordinatorimpl class mentioned above, and it will be easier to understand.

/**   * Starts running a batch for this {@link ConnectorCoordinator} if a batch is   * not already running.   *   * @return true if this call started a batch   * @throws ConnectorNotFoundException if this {@link ConnectorCoordinator}   *         does not exist.   */  //@Override  public synchronized boolean startBatch() throws ConnectorNotFoundException {    verifyConnectorInstanceAvailable();    if (!shouldRun()) {      return false;    }    BatchSize batchSize = loadManager.determineBatchSize();    if (batchSize.getMaximum() == 0) {      return false;    }    taskHandle = null;    currentBatchKey = new Object();    try {      BatchCoordinator batchCoordinator = new BatchCoordinator(this);      TraversalManager traversalManager =          getConnectorInterfaces().getTraversalManager();      Traverser traverser = new QueryTraverser(pusherFactory,          traversalManager, batchCoordinator, name,          Context.getInstance().getTraversalContext());      TimedCancelable batch =  new CancelableBatch(traverser, name,          batchCoordinator, batchCoordinator, batchSize);      taskHandle = threadPool.submit(batch);      return true;    } catch (ConnectorNotFoundException cnfe) {      LOGGER.log(Level.WARNING, "Connector not found - this is normal if you "          + " recently reconfigured your connector instance: " + cnfe);    } catch (InstantiatorException ie) {      LOGGER.log(Level.WARNING,          "Failed to perform connector content traversal.", ie);      delayTraversal(TraversalDelayPolicy.ERROR);    }    return false;  }

The method first instantiates the batchcoordinator class, And the batchcoordinator = new batchcoordinator (this). From its class name, we can guess that it is a coordinator role.

Traverser = new querytraverser (pusherfactory, traversalmanager, batchcoordinator, name, context. getinstance (). gettraversalcontext ());

When querytraverser is instantiated, batchcoordinator acts as a class of the traversalstatestore interface type for storing the connector status;

Timedcancelable batch = new cancelablebatch (traverser, name, batchcoordinator, batchcoordinator, batchsize );

When cancelablebatch is instantiated, The batchcoordinator parameter 1 acts as the batchresultrecorder interface type class, which is used to record the connector collection result information;

The batchcoordinator parameter 2 acts as the batchtimeout interface type class and is generally used to forcibly reset the connector instance.

Familiarize yourself with the batchcoordinator class source code and see how it implements the functions of the above three types of interfaces:

/** * Coordinate operations that apply to a running batch with other changes that * affect this [@link {@link ConnectorCoordinatorImpl}. * <p> * The {@link ConnectorCoordinatorImpl} monitor is used to guard batch * operations. * <p> * To avoid long held locks the {@link ConnectorCoordinatorImpl} monitor is * not held while a batch runs or even between the time a batch is canceled * and the time its background processing completes. Therefore, a lingering * batch may attempt to record completion information, modify the checkpoint * or timeout after the lingering batch has been canceled. These operations * may even occur after a new batch has started. To avoid corrupting the * {@link ConnectorCoordinatorImpl} state this class employs the batchKey * protocol to disable completion operations that are performed on behalf of * lingering batches. Here is how the protocol works. * <OL> * <LI>To start a batch starts while holding the * {@link ConnectorCoordinatorImpl} monitor assign the batch a unique key. * Store the key in ConnectorCoordinator.this.currentBatchKey. Also create a * {@link BatchCoordinator} with BatchCoordinator.requiredBatchKey set to the * key for the batch. * <LI>To cancel a batch while holding the ConnectorCoordinatorImpl monitor, * null out ConnectorCoordinator.this.currentBatchKey. * <LI>The {@link BatchCoordinator} performs all completion operations for a * batch and prevents operations on behalf of non current batches. To check * while holding the {@link ConnectorCoordinatorImpl} monitor it * verifies that * BatchCoordinator.requiredBatchKey equals * ConnectorCoordinator.this.currentBatchKey. * </OL> */class BatchCoordinator implements TraversalStateStore,    BatchResultRecorder, BatchTimeout {  private static final Logger LOGGER =      Logger.getLogger(BatchCoordinator.class.getName());  private final Object requiredBatchKey;  private final ConnectorCoordinatorImpl connectorCoordinator;  /**   * Creates a BatchCoordinator   */  BatchCoordinator(ConnectorCoordinatorImpl connectorCoordinator) {    this.requiredBatchKey = connectorCoordinator.currentBatchKey;    this.connectorCoordinator = connectorCoordinator;  }  public String getTraversalState() {    synchronized (connectorCoordinator) {      if (connectorCoordinator.currentBatchKey == requiredBatchKey) {        try {          return connectorCoordinator.getConnectorState();        } catch (ConnectorNotFoundException cnfe) {          // Connector disappeared while we were away.          throw new BatchCompletedException();        }      } else {        throw new BatchCompletedException();      }    }  }  public void storeTraversalState(String state) {    synchronized (connectorCoordinator) {      if (connectorCoordinator.currentBatchKey == requiredBatchKey) {        try {          connectorCoordinator.setConnectorState(state);        } catch (ConnectorNotFoundException cnfe) {          // Connector disappeared while we were away.          // Don't try to store results.          throw new BatchCompletedException();        }      } else {        throw new BatchCompletedException();      }    }  }  public void recordResult(BatchResult result) {    synchronized (connectorCoordinator) {      if (connectorCoordinator.currentBatchKey == requiredBatchKey) {        connectorCoordinator.recordResult(result);      } else {        LOGGER.fine("Ignoring a BatchResult returned from a "            + "prevously canceled traversal batch.  Connector = "            + connectorCoordinator.getConnectorName()            + "  result = " + result + "  batchKey = " + requiredBatchKey);      }    }  }  public void timeout() {    synchronized (connectorCoordinator) {      if (connectorCoordinator.currentBatchKey == requiredBatchKey) {        connectorCoordinator.resetBatch();      } else {        LOGGER.warning("Ignoring Timeout for previously prevously canceled"            + " or completed traversal batch.  Connector = "            + connectorCoordinator.getConnectorName()            + "  batchKey = "+ requiredBatchKey);      }    }  }  // TODO(strellis): Add this Exception to throws for BatchRecorder,  //     TraversalStateStore, BatchTimeout interfaces and catch this  //     specific exception rather than IllegalStateException.  private static class BatchCompletedException extends IllegalStateException {  }}

 

From the code above, we can see that the batchcoordinator class mainly uses the dependent connectorcoordinatorimpl class member and calls the corresponding method of the connectorcoordinatorimpl class. This processing method is a bit similar to the decoration mode. This article will write it here, leave the remaining parts for further analysis.

---------------------------------------------------------------------------

Connector connector for enterprise search engine development in this series is self-developed

Reprinted please indicate the source of the blog garden hedgehog gentle

This article link http://www.cnblogs.com/chenying99/archive/2013/03/18/2965328.html