net.sf.saxon.functions

## Class Tokenize

- java.lang.Object
- - net.sf.saxon.expr.Expression
  - - net.sf.saxon.expr.FunctionCall
    - - net.sf.saxon.functions.SystemFunctionCall
      - - net.sf.saxon.functions.Tokenize

- All Implemented Interfaces:
    Serializable, SourceLocator, LocationProvider, SaxonLocator, Callable, InstructionInfo, Locator

      public class Tokenize
      extends SystemFunctionCall
      implements Callable

  This class implements the tokenize() function for regular expression matching. This returns a sequence of strings representing the unmatched substrings: the separators which match the regular expression are not returned.
  See Also:
    Serialized Form

- - ### Field Summary

    - ### Fields inherited from class net.sf.saxon.functions.SystemFunctionCall

      `operation`

    - ### Fields inherited from class net.sf.saxon.expr.FunctionCall

      `argument`

    - ### Fields inherited from class net.sf.saxon.expr.Expression

      `EFFECTIVE_BOOLEAN_VALUE, EVALUATE_METHOD, EVENT_FEED_METHOD, INHERITED_CONTEXT, INSPECTION_CONTEXT, ITEM_FEED_METHOD, ITERATE_METHOD, locationId, MAX_SEQUENCE_LENGTH, MAX_STRING_LENGTH, NAVIGATION_CONTEXT, NODE_VALUE_CONTEXT, PROCESS_METHOD, PUSH_SELECTION, staticProperties, UNBOUNDED_LOWER, UNBOUNDED_UPPER, W3C_CONSUMING, W3C_FREE_RANGING, W3C_GROUP_CONSUMING, W3C_MOTIONLESS, WATCH_METHOD`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Tokenize()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `Sequence` | `call(XPathContext context, Sequence[] arguments)` Evaluate the expression
    `RegularExpression` | `getCompiledRegularExpression()` Get the compiled regular expression if available, otherwise return null
    `SequenceIterator` | `iterate(XPathContext c)` Iterate over the results of the function
    `Expression` | `optimize(ExpressionVisitor visitor, ExpressionVisitor.ContextItemType contextItemType)` Perform optimisation of an expression and its subexpressions.
    `Expression` | `simplify(ExpressionVisitor visitor)` Simplify and validate.

    - ### Methods inherited from class net.sf.saxon.functions.SystemFunctionCall

      `addContextDocumentArgument, addDocToPathMap, bindStaticContext, checkArguments, computeCardinality, computeSpecialProperties, copy, equalOrNull, equals, getContextNode, getConvertingCallable, getDefaultArgumentNode, getDetails, getErrorCodeForTypeErrors, getFunctionItemType, getImplementationMethod, getItemType, getOperation, getRequiredType, iterateSubExpressionInfo, makeSystemFunction, setDetails, useContextItemAsDefault`

    - ### Methods inherited from class net.sf.saxon.expr.FunctionCall

      `addExternalFunctionCallToPathMap, checkArgumentCount, explain, getArguments, getDisplayName, getExpressionName, getFunctionName, getNumberOfArguments, hashCode, iterateSubExpressions, preEvaluate, promote, replaceSubExpression, setArguments, setFunctionName, simplifyArguments, toString, typeCheck`

    - ### Methods inherited from class net.sf.saxon.expr.Expression

      `addToPathMap, adoptChildExpression, checkForUpdatingSubexpressions, checkPermittedContents, computeDependencies, computeStaticProperties, doPromotion, dynamicError, effectiveBooleanValue, evaluateAsString, evaluateItem, evaluatePendingUpdates, explain, getCardinality, getColumnNumber, getColumnNumber, getConstructType, getContainer, getDependencies, getEvaluationMethod, getExecutable, getExpressionCompiler, getHostLanguage, getIntegerBounds, getIntrinsicDependencies, getLineNumber, getLineNumber, getLocationId, getLocationProvider, getObjectName, getProperties, getProperty, getPublicId, getSlotsUsed, getSpecialProperties, getStreamability, getStreamingAdjunct, getSystemId, getSystemId, hasVariableBinding, implementsStaticTypeCheck, isSubtreeExpression, isUpdatingExpression, isVacuousExpression, iterateEvents, markTailFunctionCalls, process, resetLocalStaticProperties, setContainer, setEvaluationMethod, setFiltered, setFlattened, setLocationId, staticTypeCheck, suppressValidation, toPattern, toStreamingPattern, typeError`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### Tokenize

          public Tokenize()

  - ### Method Detail

    - #### simplify

          public Expression simplify(ExpressionVisitor visitor)
                              throws XPathException

      Simplify and validate. This is a pure function so it can be simplified in advance if the arguments are known
      Overrides:
        `simplify` in class `FunctionCall`
      Parameters:
        `visitor` - an expression visitor
      Returns:
        the simplified expression
      Throws:
        `XPathException` - if an error is discovered during expression rewriting

    - #### optimize

          public Expression optimize(ExpressionVisitor visitor,
                            ExpressionVisitor.ContextItemType contextItemType)
                              throws XPathException

      Perform optimisation of an expression and its subexpressions.

      This method is called after all references to functions and variables have been resolved to the declaration of the function or variable, and after all type checking has been done.

      Overrides:
        `optimize` in class `SystemFunctionCall`
      Parameters:
        `visitor` - an expression visitor
        `contextItemType` - the static type of "." at the point where this expression is invoked. The parameter is set to null if it is known statically that the context item will be undefined. If the type of the context item is not known statically, the argument is set to `Type.ITEM_TYPE`
      Returns:
        the original expression, rewritten if appropriate to optimize execution
      Throws:
        `XPathException` - if an error is discovered during this phase (typically a type error)

    - #### getCompiledRegularExpression

          public RegularExpression getCompiledRegularExpression()

      Get the compiled regular expression if available, otherwise return null
      Returns:
        the compiled regular expression, or null

    - #### iterate

          public SequenceIterator iterate(XPathContext c)
                                   throws XPathException

      Iterate over the results of the function
      Overrides:
        `iterate` in class `Expression`
      Parameters:
        `c` - supplies the context for evaluation
      Returns:
        a SequenceIterator that can be used to iterate over the result of the expression
      Throws:
        `XPathException` - if any dynamic error occurs evaluating the expression

    - #### call

          public Sequence call(XPathContext context,
                      Sequence[] arguments)
                        throws XPathException

      Evaluate the expression
      Specified by:
        `call` in interface `Callable`
      Parameters:
        `context` - the dynamic evaluation context
        `arguments` - the values of the arguments, supplied as SequenceIterators
      Returns:
        the result of the evaluation, in the form of a SequenceIterator
      Throws:
        `XPathException` - if a dynamic error occurs during the evaluation of the expression

Copyright (c) 2004-2013 Saxonica Limited. All rights reserved.