SOA and OSB Basics

Transient BPEL Process

Transient processes do not incur dehydration during their process execution. If an executing process experiences an un-handled fault or the server crashes, instances of a transient process do not leave a trace in the system. Thus, these instances cannot be saved in-flight regardless if they complete normally or abnormally. Transient processes are typically short-lived, request-response style processes. Synchronous processes are examples of transient processes.

Durable BPEL Process

Durable processes incur one or more dehydration points in the database during execution. Dehydration is triggered by one of the following activities:

• Receive activity
• OnMessage branch in a pick activity
• OnAlarm branch in a pick activity
• Wait activity
• Reply activity
• checkPoint() within a <bpelx:exec> activity

Durable processes are typically long-living and initiated through a one-way invocation. Because of out-of-memory and system downtime issues, durable processes cannot be memory-optimized.

XA Transaction

• An XA transaction, in the most general terms, is a "global transaction" that may span multiple resources.
• An XA transaction involves a coordinating transaction manager, with one or more databases (or other resources, like JMS) all involved in a single global transaction.
• XA transactions come from the X/Open group specification on distributed, global transactions. JTA includes the X/Open XA spec, in modified form.

Non-XA Transaction

• A non-XA transaction always involves just one resource.
• Non-XA transactions have no transaction coordinator, and a single resource is doing all its transaction work itself (this is sometimes called local transactions).
• Most stuff in the world is non-XA - a Servlet or EJB or plain old JDBC in a Java application talking to a single database.

targetNamespace

targetNamespace is the namespace that is going to be assigned to the schema you are creating. It is the namespace an instance is going to use to access the types it declares. In the following code, the schema will be assigned to the namespace http://www.example.com/test/po

<schema xmlns="http://www.w3.org/2001/XMLSchema"

targetNamespace="http://www.example.com/test/po">

...

</schema>

defaultNamespace

xmlns=”” or namespace– Defines the default namespace within the current document for all non-prefixed elements. XML Schema is an XML document, it is then possible to define a default XML namespace for the XML file itself (this is what xmlns attribute does); the implications are multiple: authoring, and composition. For example, one does not have to use a prefix for the items defined in the schema, that are later on referenced elsewhere in the same file (e.g. a global simpleType used as a type for an attribute or element).

<purchaseOrder xmlns="http://www.example.com/test/po"

xmlns:addr="http://www.example.com/test/po/addr">

  <accountName>abc</accountName>

  <accountNumber>12345</accountNumber>

  <addr:street>xyz</addr:street>

</purchaseOrder>

In an XML document instance, you declare the namespaces you are going to be using by means of the xmlns attribute.

The default namespace here is  http://www.example.com/test/po, which makes reference to the schema previously created. This namespace applies to the element that declares it, and its child elements, unless they are prefixed. In the example, all the elements belong to the default namespace, except addr:street. Since it is prefixed, it belongs to the addr namespace (xmlns:addr=" http://www.example.com/test/po/addr")

When to go for SOA & When to go for OSB

• For service virtualization and brokering activities use OSB
• For long running and stateful orchestrated tasks, use Oracle BPEL
• For automate business activities based on a process definition use Oracle BPEL
• For incorporating Human Workflow and/or Oracle Business Rules, use Oracle BPEL PM
• For applying policy-centralization and reliable-messaging techniques on web-services, use OSB
• For stateless and short-lived web-service orchestrations use OSB
• For synchronous entity-based services or pass-through operations use OSB

Conclusion: Based on individual business needs decision should be made. 

xsd:import: 

• It is used to refer to declarations or definitions that are in a different target namespace. 
• The import element is used to add multiple schema's with different target namespace to a document.

xsd:include: 

• It is used to refer to declarations or definitions that are in the same target namespace. 
• Include element is used to add multiple schemas with same target namespace to a document.

Note: If the schema target namespace don't match, include won't work.

Pipeline Components (Nodes)

Start Node - Every pipeline begins with a start node. All messages enter the pipeline through the start node, and all response messages are returned to the client through the start node. There is nothing to configure in a start node.

Pipeline Pair Node - A pipeline pair node combines a single request pipeline and a single response pipeline in one top-level element. A pipeline pair node can have only one direct descendant in the pipeline. During request processing, only the request pipeline is executed when Service Bus processes a pipeline pair node. The execution path is reversed when Service Bus processes the response pipeline.

Stage - Request pipelines, response pipelines, and error handlers can contain stages, where you configure actions to manipulate messages passing through the pipeline.

Error Handler - An error handler can be attached to any node or stage, to handle potential errors at that location.

Operational Branch - Operational branching is supported for WSDL-based services, where the branching is based on operations defined in the WSDL file.

Conditional Branch - Conditional branching is supported for conditions defined in an XPath-based switch table.

Route - A route node performs request/response communication with another service or component. It represents the boundary between request and response processing for the pipeline. When the route node dispatches a request message, the request processing is considered complete. When the route node receives a response message, the response processing begins. The route node supports conditional routing as well as request and response transformations. Because a route node represents the boundary between request and response processing, it cannot have any descendants in the pipeline

• Last node in request processing.  It can be thought of as a bridge between request pipeline processing and the response pipeline processing.
• You can only execute one route in your Proxy Service.
• Can only be created in a route node.
• OSB will wait for the Route call to finish before continuing to process.
• If you are calling a Business service and you specify Best Effort for QoS (Quality of Service), then OSB will release the thread it is holding while the business service executes.
• If you are calling a Business service and you specify Exactly Once or At Least Once for QoS, then OSB will hold onto the thread while the business service executes.
• If you are calling a local Proxy service, then OSB will hold onto the thread until the Proxy service finishes executing.

Rest Branch - Since Un-Typed REST services does not use WADL, we can’t use Operational Branch as above. So in this release, OSB introduced a new node called  REST Branch for this purpose.

Route Actions

Dynamic Routing - It is a kind of the Content-based Routing pattern, which is used when the Business Service endpoint is required to be determined at run time. Assign a route for a message based on routing information available in an XQuery resource

Routing - It is used when the Business Service endpoint is established during design time. Identify a target service for the message and configure how the message is routed to that service

Routing Table - A routing table is a set of routes wrapped in a switch-style condition table. It is a short-hand construct that allows different routes to be selected based upon the results of a single XQuery expression.

Communication Actions

Service Callout

• Can have multiple Service Callout nodes in a Proxy service.
• Pipeline processing will continue after a Service Callout.
• Can be invoked from the request and/or response pipelines.
• Used to enrich the incoming request or outgoing response. For example, a call to get a country code.
• Used for real time request/response calls (Synchronous calls).
• OSB will hold a thread and not continue until the Service Callout completes.
• Can tie up resources and degrade performance under heavy loads.
• Well, Service Callout in OSB is similar to Invoke activity in SOA. We use Invoke activity in SOA to trigger external services like (WS Service, DB Service, JMS Service, FTP Service etc). Similarly we use Service Callout in OSB to trigger external services.
• Similarities & differences b/w Invoke & Service Callout: Both used to trigger external services, but Invoke can be used for both Synchronous& Asynchronous external services (i.e the calling external service may or may not give the response). But Service Callout is used only to trigger Synchronous external services. To trigger one way services there is another component in OSB which is ‘Push’ which also works same as Service Callout, only difference is Push is used for triggering One way external services and Service Callout is used for triggering Synchronous external services. Invoke activity will be having ‘Input Variable’ and ‘Output variable’ and similarly Service Callout will also has ‘Request Variable’ & ‘Response Variable’.

Publish

• Can be synchronous or asynchronous
• If you are calling a business service with a Quality of Service of Best Effort , then it will be an asynchronous call.
• If you call a business service with a Quality of Service of Exactly Once or At Least Once, OSB will wait until the processing completes in the business service completes before proceeding and it is effectively a synchronous call.
• If you are calling a local proxy service, OSB will wait until the processing in the local proxy service completes and it is effectively a synchronous call.
• Can be invoked from the request and/or response pipelines.
• Best to use when you do not need to wait for a response from the process you are calling (Fire and Forget.... Asynchronous Calls).

Dynamic Publish - Publish a message to a service specified by an XQuery expression.

Publish Table - Publish a message to zero or more statically specified services. Switch-style condition logic is used to determine at runtime which services will be used for the publish.

Routing Options - Modify any or all of the following properties in the outbound request: URI, Quality of Service, Mode, Retry parameters, Message Priority.

Transport Headers - Used to set the header values in messages.

Message Processing Actions

Assign - Assign the result of an XQuery or XSLT expression to a context variable.

Delete - Delete a context variable or a set of nodes specified by an XPath expression

Insert - Insert the result of an XQuery or XSLT expression at an identified place relative to nodes selected by an XPath expression.

Java Callout - Invoke a Java method from within the pipeline.

MFL Translate - Convert message content from XML to non-XML, or vice versa, in the message pipeline. An MFL is a specialized XML document used to describe the layout of binary data. It is an Oracle proprietary language used to define rules to transform formatted binary data into XML data, or vice versa.

nXSD Transalte - Convert message content from XML to native format data, or vice versa, in the message pipeline.

Rename - Rename elements selected by an XPath expression without modifying the contents of the element.


Replace - Replace a node or the contents of a node specified by an XPath expression. The node or its contents are replaced with the value returned by an XQuery expression. A replace action can be used to replace simple values, elements and even attributes. An XQuery expression that returns nothing is equivalent to deleting the identified nodes or making them empty, depending upon whether the action is replacing entire nodes or just node contents. The replace action is one of a set of Update actions.

Validate - Validate elements selected by an XPath expression against an XML schema element or a WSDL resource. You can validate global elements only; Service Bus does not support validation against local elements.

Reporting Actions

Alert - Generate alerts based on message context in a pipeline, to send to an alert destination. Unlike SLA alerts, notifications generated by the alert action are primarily intended for business purposes, or to report errors, and not for monitoring system health. Alert destination should be configured and chosen with this in mind. If pipeline alerting is not enabled for the service or enabled at the domain level, the configured alert action is bypassed during message processing

Log - Construct a message to be logged and to define a set of attributes with which the message is logged.

Report - Enable message reporting for a pipeline. An XQuery/XSLT expression is used to create the data that is reported to the Service Bus dashboard.You use key value pairs to extract key identifiers from any message context variable or message payload, and ignore the rest of the message.


Flow Control Actions

For Each - Iterate over a sequence of values and execute a block of actions.

If..Then - Perform an action or set of actions conditionally, based on the Boolean result of an XQuery expression.

Raise Error - Raise an exception with a specified error code (a string) and description.

Reply - Specify that an immediate reply be sent to the invoker. The reply action can be used in the request, response or error pipeline. You can configure it to result in a reply with success or failure. In the case of reply with failure where the inbound transport is HTTP, the reply action specifies that an immediate reply is sent to the invoker

Resume - Resume message flow after an error is handled by an error handler. This action has no parameters and can only be used in error handlers.

Skip - Specify that at runtime, the execution of this stage is skipped and the processing proceeds to the next stage in the pipeline. This action has no parameters and can be used in the request, response or error pipelines.

Thank You!!!!

Happy Learning!!!!

Oracle Service Bus (OSB) Best Practices

XQuery

1)      Use of Double Slash

·         Use of double slashes should be avoided to enable better code efficiency.

·         Use of “//” will force the entire payload to be read and parsed.

·         While double slashes ‘//’ imply all the occurrences of the required node, single slash ‘/’ will ensure the exact node that is required.

Recommendation: Use "//" only if the exact location of a node is not known at design time.

Example: “/Order/Address” instead of “/Order//Address”

2)      Indexing the XPath statements wherever possible also increase the code efficiency. By adding a simple [1], we can index the XPath statements.

Example: “$body/Order [1]/Address [1]” instead of “$body/Order/Address”

Note: Do not index when you need a whole array of nodes returned. Indexing will only return   the first item node of the array.

3)      Extract frequently used parts of a large XML document as intermediate variables within a FLWOR expression.

OR

If you want to use any intermediate XPath continuously, then assign that to a variable and use it rather than querying it multiple times.

 

An intermediate variable can be used to store the common context for multiple values.

Example:

$body/Order[1]/Summary[1]/Total,$body/Order[1]/Summary[1]/Status,                                                $body/Order [1]/Summary [1]/Shipping

The above XPaths can be changed to use an intermediate variable:

               let $summary: = $body/Order [1]/Summary [1]

               $summary/Total, $summary/Status, $summary/Shipping

Recommendation: Use of intermediate variables will consume more memory, but will reduce redundant XPath processing.

                                                                          

4)      Use FLOWR expressions wherever possible

Example:

Normal Syntax	FLOWR Syntax
doc("books.xml")/bookstore/book[price>30]/title	For $x in doc("books.xml")/bookstore/book where $x/price>30 return $x/title

5)      Code should not have any warnings or errors

6)      Looping

minOccurs	maxOccurs	forLoop
0	1	NO
1	1	NO
1	Unbounded	YES
0	Unbounded	YES

7)      Check for NULL values before assigning mandatory fields.

8)      XQuery Transformation:

a.       XQuery mapper may be more productive (faster) to use than coding a sequence of low level actions.

b.      An XQuery subroutine may be less efficient than using the low level built in transformation actions such as Rename, Delete, etc.

9)      Reduce the number of parameters passing to the XQuery. Instead you can pass a single

Xml which will have all the parameters mapped.

Reason: Used for effective optimization.

10)   Declare main function of the XQuery with proper name i.e. with Name of an XQuery.

11)   Update actions like Delete, Insert, Replace, Rename are more efficient for minor fixes to a document than using Assign with an XQuery that regenerates the entire document, especially if the document is large.

12)   Split large sized DB Query xml’s to multiple parts so that XQuery parsing will be easy. 

Reason: Design the DB entries in such a way that you would be easily able to split with respect to some parameters.

13)   Avoid declaring namespaces above XQuery. It will lead to different namespace prefixes during runtime. Instead you can put the namespace inline with XQuery xml tags.

14)   Use Fully Qualified XPath expressions.

Example:

Use /bookstore/ns0:book/ns1:title instead of /bookstore/*:book/*:title

Reason: This will fetch valid data in case we have same elements in different schemas.

Should avoid namespace wildcards like /*: book/*: title

Reason: For better Performance and clarity.

Proxy Service

15)   Avoid creating many OSB context variables that are used just once within another XQuery.

Recommendation: Avoiding redundant context variable creation eliminates overheads associated with internal data format conversions. This benefit has to be balanced against visibility of the code and reuse of the variables.

16)   It is recommended to use $body/*[1] in any XQuery/XSLT input as it enables faster execution without invoking XQuery engine.

Reason: A general XPath like "$body/Order" must be evaluated by the XQuery engine before the primary transformation resource is executed. OSB treats "$body/*[1]" as a special XPath that can be evaluated without invoking the XQuery engine.

17)   Transforming contents of a context variable such as $body could be time-consuming.

Recommendation:

·         Use a Replace action to complete the transformation in a single step.

·         If the entire content of $body is to be replaced, leave the XPath field blank and select "Replace node contents". This is faster than pointing to the child node of $body (e.g. $body/Order) and selecting "Replace entire node".

·         Leaving the XPath field blank eliminates an extra XQuery evaluation.

18)   Strive to have a single message representation for as much of the pipeline as possible.

                                    

Reason: This makes it easy to insert things at various points since you don’t have to think about what the current document format is.

19)   If a proxy service has a WSDL with multiple operations, operational branching is used to handle messages separately for each operation.

Example:

In this example CustomerPS Proxy Service has three matching operations:

GetAllCustomers, GetCustomerById and AddCustomer

                                                     

These will be routed to their equivalents using an Operational Branch. Right-click the "CustomerPS" icon and select insert into Operational Branch to branch the incoming request based on the invoked operation to the corresponding operation of the Business Service.

20)   Batch file processing: For large but low priority jobs that arrive over immediate transports like HTTP, consider configuring an HTTP proxy to accept incoming documents and publish them to a local directory using a file-based business service. Separately configure a file-based proxy that polls the directory after-hours and processes the files.

21)   If a proxy service is of type Any SOAP or Any XML, you can use a stage action to determine what the message type is, and use a conditional branching node in the flow to separate processing for each message type received.

22)   Conditional branching can be used to expose the routing alternatives at the top level flow view.

Example:

If you invoke service A or service B based on a condition, instead of configuring this conditional branching within the route node, you can expose or highlight this branching in the message flow itself, and use simple route nodes as the sub flows for each of the branches. This is a style decision that you must make. Note that the approach of configuring the route node within each of the branches can get awkward if the fan-out of messages from the branch is large.

23)   Use “local” transport protocol for “internal services”.

Unit Testing:

·         Do nothing i.e. don’t unit test them just test with other services which may not be that bad as they are like internal services.

·         Change the protocol in Dev environments allowing them to be tested individually.

·         By adding SOAP based wrappers for services with careful governance.

24)   Disable or delete all log actions.

Reason: Log actions add an I/O overhead. Logging also involves an XQuery evaluation which can be expensive. Writing to a single device (resource or directory) can also result in lock contentions.

25)   Set the appropriate QOS level and transaction settings.

Do not set XA or Exactly-Once unless the reliability level required is once and only once and it’s possible to use the setting (it is not possible if the client is a HTTP client).

Reason: If OSB initiates a transaction, it is possible to replace XA with LLR to achieve the same level of reliability.

OSB can invoke a back end HTTP service asynchronously if the QOS is "Best- Effort".

Reason: Asynchronous invocation allows OSB to scale better with long running back-end services. It also allows Publish over HTTP to be truly fire-and-forget.

26)   When publishing or routing to multiple services use the table version of those actions.

Reason: A table is typically more efficient than a chained if/then/else.

2.3 DO’s and DON’T’s

27)   Users with console write access should never share a userid because console sessions are managed per userid.

Reason: If two users that use the same userid make changes in the console, each could affect the changes the other is making.

28)   DON’T try to develop your entire solution only on OSB

Reason: OSB is not alone it’s a part of Oracle SOA Suite, though it’s a general misconception since it’s available as a separate installer through OTN. Some Project makes mistake by developing their entire solution on OSB.

29)   DON’T design state full transactions within OSB

Reason: OSB is a stateless product it’s not design to handle state-full transactions. For state-full transaction one should use BPEL or BPM instead of OSB.

30)   DON’T use OSB to execute complex SQL, SQL Procedures

Reason: Though OSB provide mechanism to execute SQL and SQL procedure within its pipeline but it needs to be restricted only for retrieve (select) operation. NEVER try to execute SQL procedure or insert statements within OSB pipeline.

31)   DON’T develop complex business logic within OSB’s message pipeline

Reason: Oracle SOA Suite has OBR (Oracle Business Rule) component to define business logic than why to define business rules somewhere else in OSB. Define business rule in OBR expose it as web service and consume your business rule web service in OSB.

32)   DON’T use sbconsole for development:

Reason: OEPE create OSB projects and its components in local file system, which can be easily source controlled apart from that, while considering OSB development OEPE has many advantage over sbconsole like XQuery editor, split-join etc.

33)   DON’T develop entire message flow with single stage:

Reason: Each stage should have only relevant actions and the stage name should clearly explain what that stage does for other to understand without going through each and every action in that stage.

·         Multiple stages provide a natural modularity compared to configuring all the actions in a single stage.

·         Each stage in a request or response pipeline can have a separate error handling pipeline. If you design your pipeline with multiple stages, you can avoid writing a single error handler that must handle all errors by all actions in a single stage.

34)   Use Split-Join

Reason: Split-Join is used to reduce latency when CPU is not saturated. Parallel execution is a common paradigm used to reduce latency while consuming higher system resources. If a set of tasks have no dependency on one another, they can be executed in parallel. This is very useful for latency sensitive use cases.

Example:

                  

35)   Use Throttling

Reason: Throttling is used to protect back-end systems from spikes in load. Throttling can be implemented creating Throttling policies for a Business Service within OSB or by using Work Managers at the WLS level. One can limit the maximum number of concurrent calls to the back-end service using Throttling.

36)    Use Service Virtualization

Reason: Create additional agility by replacing direct coupling and to provide a virtual endpoint    with Support of different message formats, move of endpoints, service versioning, better availability, scalability and Security.

37)   If you are using datasouce for select or query operations, then use non-XA driver.  

38)   Loose Coupling

 It is suggested to implement loose coupling among different projects so as to remove any dependency among them.

           

Reason: To call another or provider service, instead of directly making a service callout to the provider proxy, we can introduce an enterprise business service with the URL of the provider proxy which in turn can call the provider business service. This approach will remove the dependency among different projects.

 

3 Performance

39)   Enable Streaming for pure Content-Based Routing scenarios.

40)   In addition one may configure the services on the bus using high availability option so that the service may be exposed in multiple URIs and if one is not available the others will be accessed automatically.

41)   Use whatever tooling you are comfortable with. If performance is an issue use action level performance metrics to identify the actions to refactor.

42)   The default (unconditional) routing configuration has the best performance as the message is streamed without interruption.

43)   Use Service Result Caching

Reason: It is one of the options that you can use when you want to improve Oracle Service Bus    performance. Service Result caching is used when we have business service which connects to external service which returns somewhat static response. So using Service Result Caching we don’t hit external service for same request instead it will take the response from cache which improves the OSB performance.

3.1 Processing of large files

             

              While processing large files, there are certain parameters to be considered

·         Heap Memory

·         CPU Utilization

3.1.1 Heap Memory

Loading large files into memory would require enough heap size. Insufficient heap size will lead to “Out of memory” errors and might also crash the weblogic server.

In order to avoid these error, we might need to add more memory to the server. But this is not always a feasible option as the server memory also has a limit. Instead, one of the following options can be tried.

a.       Content Streaming

b.      Process in chunks

With Content Streaming, the whole object will not be loaded into memory buffer and hence, we can consume large messages. But content streaming has certain limitations because the whole object is not loaded in memory. You can find these limitations at following link and decide if you can implement your use case with these limitations.

http://download.oracle.com/docs/cd/E13159_01/osb/docs10gr3/userguide/context.html#wp1110513

Process in chunks – Another option would be to break the file into smaller chunks and then use OSB to read and process those chunks.

 3.1.2 CPU Utilization

When processing large files in OSB, just passing over the content as-is to backend system will not be an issue. But transforming the large content in OSB is the pain point. Any transformation (be it XQuery or XSLT) takes lot of CPU cycles and transformation of huge payloads will consume 100% CPU, resulting in overall drop of the performance. Hence, it is not recommended to have complex transformations and transformations of large XMLs within OSB. 

Following are few options:

a.       Opt to move transformations on large payload to backend systems. 

b.      Try for XML Appliances which are specifically meant for large transformations with little impact on CPU utilization.

3.2 Caching mechanisms in OSB

When caching data in OSB, two points should always be considered – 1) How much data needs to be cached 2) Which caching API to be used.

There are two caching mechanisms that we can look for –

1)      Java Caching System

2)      Oracle Coherence Caching             

3.2.1 Java Caching System

This is a distributed caching system written in Java. This caching system provides a means to manage cached data, thus speeding up the processing. Beyond simply caching data into memory, it provides additional features like memory management, thread pool controls, etc. This is a simple caching scheme where static data will be cached per server. This can be easily integrated with OSB. But it is not suitable for caching large amounts of data. It might fail under heavy load in an OSB.

3.2.2 Oracle Coherence Caching

This is the Oracle recommended caching mechanism where the coherence cluster should be configured in the weblogic server. This is very robust and stable cache mechanism which can handle heavy load and large amount of data. The cached data will be available throughout the cluster.