J2EE integration module

From ClickTale Wiki

The instructions on this page are intended for self service and online accounts.
Enterprise customers should contact their account managers for integration support.

Introduction

ClickTale is happy to announce the availability of the J2EE (Java Platform, Enterprise Edition) integration module. Based on the same architecture as ASP.NET integration module and PHP integration module, the J2EE module allows full integration of ClickTale with dynamic, session-based, behind-a-login and POST-processing pages.

The module is composed of a filter, a servlet and a test page and is added to the website as any other library.

Technical notes

If you are using struts version 2 and above, you may need to edit struts.xml and add a line to exclude our servlet from being processed by struts. See the line below:

For older struts versions:

Struts 1.3 - no changes to struts configuration are required.
Customers with JDK 1.5 should use the J2EE integration module version compiled with jdk 1.4 found below – tested and works

This Integration module is for J2EE sites using dynamic content which changes for each user independently. For example, shopping carts are often such pages. It will allow you to store the exact version that the user saw and allow our fetching bot to collect that version so it will be available to you in the playback of visitors visits and other reports. The J2EE Integration Module adds further support to cases that were not previously covered, such as POST processing pages.

The module allows better integration of a J2EE site with ClickTale by enabling accurate caching of the HTML that is sent to the visitor in ClickTale recorded pageviews.

Installation Guide

Remove the existing ClickTale code and/or disable other integration methods (for example, J2EE integration code - which supports session state only).
Download the distribution of the latest version of the ClickTale J2EE Integration module that suits your environment the most:
1. for JVM 1.6 and above
2. for JVM 1.6 and above with dependances - all referenced libraries packaged into single JAR package for quick deployment.
3. for older JVM (1.4 and 1.5)
Generate the ClickTale tracking code you want to use in your pages (or locate your already generated code).
Decompress the 2 archive files, and copy to the site location.
Locate the WEB-INF/ClickTaleScripts.xml file that comes in the clicktale-sample.war file and open it in a text editor. Paste your ClickTale Tracking Code from the previous step into the appropriate CDATA fields into the ClickTaleScripts.xml file.
In the bottom script before the call to ClickTale(###) function add the following line:
```
ClickTaleFetchFrom="http://%RootPath%/ClickTaleCache?h=%CacheToken%";
```

Your xml file should now look like this (for the regular, https/SSL compliant version of the tracking code): :

<?xml version="1.0" encoding="utf-8" ?>
<!-- Put this in the root of the web site -->
<scripts>
  <script name="Top">
    <![CDATA[
<!-- ClickTale Top part -->
<script type="text/javascript">
var WRInitTime=(new Date()).getTime();
</script>
<!-- ClickTale end of Top part -->
]]>
  </script>
  <script name="Bottom" DoNotReplaceCondition="&lt;!-- No ClickTale --&gt;|&lt;!-- ClickTale Bottom part --&gt;" InsertBefore="&lt;!-- ClickTale Bottom part Here --&gt;|&lt;/body&gt;">
    <![CDATA[
<!-- ClickTale Bottom part -->
<div id="ClickTaleDiv" style="display: none;"></div>
<script type='text/javascript'>
document.write(unescape("%3Cscript%20src='"+
 (document.location.protocol=='https:'?
  'https://clicktale.pantherssl.com/':
  'http://s.clicktale.net/')+
 "WRb6.js'%20type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
if(typeof ClickTale=='function')
{
	var ClickTaleSSL=1;
	ClickTaleFetchFrom="http://%RootPath%/ClickTaleCache?h=%CacheToken%";
	ClickTale([Your ClickTale parameters]);
}
</script>
<!-- ClickTale end of Bottom part -->
]]>
  </script>
</scripts>

IMPORTANT NOTE! This is how the code looks like in a text editor. In a browser, "<" will be converted to "<" and ">" will become ">"

Locate and copy the various configuration files in the archive (WEB-INF and WEB-INF\classes) to your site. If you already using log4j or ehcache, please mind to integrate those instead of overwriting your files.
Locate and copy the various WEB-INF\lib files in the archive to your site.
Modify your WEB-INF\web.xml file to include the filter and servlet declarations as in the web.xml file that is included in the archive.
Copy the 'clicktale' directory to your site. This folder contains test pages that you can use to verify the correct operation of the module. You may remove the folder from production system.

Optionally: see here to disable or enable yourself from being recorded. Apply this to everyone who is working with you on the site otherwise they will use recording quota, or enable this for testing to make sure all is working correctly.

Integration Module settings

The ClickTale J2EE Integration Module settings can be configured using the clicktale.properties file located in the WEB-INF folder. The list below documents all the configuration options, including their default values. The default values are used when the configuration options is not specified in the clicktale.properties file.

ScriptsFile - relative path (local url) to the xml file that contains the java scripts to be injected into the served pages
default value: ClickTaleScripts.xml
DoNotProcessCookieName - ClickTale's web recorder ID cookie name
default value: WRUID
DoNotProcessCookieValue - value of the clicktale's web recorder ID cookie that specifies that the recording should not be performed.
default value: 0
Disable - provides an easy way to disable the clicktale filter and the caching without the need to restart the application.
default value: false
AllowedAddresses - limitation of ip addresses to serve the cached pages to. should be set to 75.125.82.64/26,50.97.162.64/26
default value: none
IPAddressHeaderFieldName - in case of presence of reverse proxy, the setting below should be used to change the default behavior of the allowed addresses check normally, the ip address of the source computer will be checked against the list, but in case of reverse proxy, usually X-Forwarded-For http header is used
default value: X-Forwarded-For
DeleteAfterPull - determines if the cached page will be immediately removed from cache when the clicktale fetcher bot requests it
default value: true
IgnoreHttpStatusCode - by default, in case the http request processing chain sets the HTTP status code to anything other than 200, the filter will not cache the page nor perform the insertation if the below line is uncommented, the behavior is changed and the filter will perform the injection regardless of the http status code set by the http request processing chain.
default value: true
CacheProvider - to use ehcache cache, set this option to com.clicktale.cache.impl.EhCacheProviderImpl. to use hash map cache, set this option to com.clicktale.cache.impl.HashMapProviderImpl
default value: none
EhCacheName - cache name in the ehcache to use. recommended value clickTaleCache. relevant only in case of ehcache based cache.
default value: none
MaxCachedPages - the maximum number of the cached pages. relevant onlu in case of hash map based cache.
default value: 100

Troubleshooting

Enabling Logging

After the files are copied you can navigate your browser to http://yoursite/clicktale/index.html to view some helpful information regarding the caching and other configurations. Changing info to debug in the log4j.properties file will output debug info to the console, this can be sent direct to a log file by editing the text to read:

log4j.rootLogger=debug, R
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.appender.R = org.apache.log4j.DailyRollingFileAppender
log4j.appender.R.File = logs/clicktale.log
log4j.appender.R.Append = true
log4j.appender.R.DatePattern = '.'yyy-MM-dd
log4j.appender.R.layout = org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern =%d [%t] %-5p %c - %m%n

Depending how the root folder is defined for your server, you might need to set the path of the log file to ../logs/clicktale.log or use another (absolute) path to enable logging correctly.

Please ensure that production servers are set back to "info" when live.

"Invalid <url-pattern>" error message

It is possible that you will receive this error message with older version of Java (1.4) or specific web servers (Tomcat 5.5, etc). The error will appear in the log files of your web server. Additionally, pages on the site related to the modified web.xml file will not load. To resolve the issue edit the web.xml file and modify the value of the <url-pattern> element for the ClickTale module such that it will be valid for your environment. We had good results replacing "*" with "/*"

Abnormal Status Code (tracking code partially injected or not injected at all)

If the module isn't injecting the tracking code all or some of the pages, this might be the result of an incorrect HTTP status codes in your application (this can be verified in the server logs, where you might see lines such as: com.clicktale.filter.ClickTaleScriptInjectFilter : abnormal status code XXX, where XXX is any status code apart from 200). To overcome this uncomment the IgnoreHttpStatusCode setting in the module's configuration file.

Configuring the caching provider

The module caches the content of the pages so it can later provide the content to ClickTale servers for processing. This caching requires some persistent storage and so different caching providers are supported. The default caching provider is the HashMap based, internal storage provider. You could also configure it to use a local ehcache for storage.

You can enable its provider by editing the ClickTale.properties file, commenting:

#CacheProvider=com.clicktale.cache.impl.HashMapProviderImpl;

and uncommenting:

CacheProvider=com.clicktale.cache.impl.EhCacheProviderImpl
EhCacheName=clickTaleCache

Caching provider settings

DeleteAfterPull = (true / false)
- If true, the cached page is deleted after clicktale fetches it from your site
MaxCachedPages = (integer)
- The size of the cache dir in MegaBytes. If the number of cached pages cache gets larger than that, stale caches are removed

Use With "Expires" Header And Enable Reuse Of Cached Pages

Some websites use "cache-control" and "expires" headers to cache pages on the client. This improves performance but may be problematic when used with our module. Pages are removed from the module's cache right after they are being accessed (for security and performance reasons). So, if a visitor browses a page more than once without refreshing the content from the server (this is usually a result of using the back button), the cache will be called several times with the same token. This will cause a cache miss for any request beyond the first one. To overcome this problem, it is possible to use the DeleteAfterPull="false" parameter in the configuration. This will disable the removal of cached pages when the ClickTale cache is called. Cached data will be removed after MaxCachedPages="???" new pages are cached, so you might want to extend this parameter as well to allow sufficient traffic between the first pageview and the next (duplicate pageview).

Q&A

Q: Are cached pages protected from access by third parties?

A: Yes, several layers of protection are in place. Only certain IPs are allowed to request the cached pages (IPs of ClickTale servers) and only processes which already have access to the page have the secret token required to request the cached content.

Q: Is it possible to inject the script in other places rather than after/before body tags?

A: Yes. By default the top script is injected after <body> and the bottom before </body>, but this can be changed by adding a InsertAfter attribute to the script[name="Top"] element or adding a InsertBefore attribute to the script[name="Bottom"] element, both are regular expressions.

Q: Some of my pages already have the ClickTale script, I do not want the script appear twice.

A: You should either remove the script from those pages and let the module handle the insertion or you should use DoNotReplaceCondition. See the code step 2 for an example. The default script in Step 2 is already configured to prevent double inclusion.

Q: After installing the module, I tried to watch a recording but I got the following notice instead: "Request from an unauthorized IP." . What should I do?

A: This could be a misconfiguration, a change in our IP addresses or a hacking attempt. Please contact us so we can investigate this further.

Q: I have installed the module but I don't see the tracking code in the source of the page. What is wrong?

A: The module will only inject the code for visitors who are classified as "to-record" (have WRUID cookie with non-zero value) or for those who are not classified (no WRUID cookie). Visitors who are classified as "not-to-record" will get no code.

Q: I'm using a proxy server for my website - what steps should I take to record properly?

A: Move the IP restriction rule to the proxy and set the IP in ClickTale.properties (located in the WEB-INF directory) to the IP of the proxy.

Q: Can I filter out specific pages from recording?

A: You may use the filter-mapping/url-pattern entry in the web.xml to set filters with some wildcard options.

Q: Can I make a programmatic decision if a page should be cached and recorded.

A: Yes - since module version 1.5. You should set the request attribute 'clickTaleDoNotProcess' to 'true' on pages you do not want to be processed.

Using a proxy server

If you have a proxy, whenever our servers will try to get the cache they will appear to have the address of the proxy which is not allowed by default. To fix this, you should set the IP address of the proxy to the AllowedAddresses value in the module's ClickTale.properties file and set a rule with your proxy to allow only our servers IP range access to the cache. Our servers' IP ranges are: 75.125.82.64/26 50.97.162.64/26 (stands for 75.125.82.64 - 75.125.82.127 and 50.97.162.64 - 50.97.162.127).

Another way to address this issue is to leverage the proxy's feature of forwarding the original IP. The proxy will normally keep the original IP address of the client in an HTTP field (most usually X-Forwarded-For). You can configure the ClickTale J2EE IM to check the AllowedAddresses setting against that field instead of against the source IP address (which will always be the proxy IP address). So, to configure the exact name of the HTTP field that determines the original IP you should add the following line to the ClickTale.properties file:

IPAddressHeaderFieldName=X-Forwarded-For

NOTE: please replace the work X-Forwarded-For with the name of the ip address header field set by the proxy.

Reference

Integration module - Good read, for a general understanding of our integration modules.