Talos Vulnerability Report

Description

A vulnerability in PDF parser of the IX SDK exists that results in out of bounds heap memory access following an unchecked memory allocation operation under specific conditions.

Tested Versions

Oracle Outside In IX sdk 8.5.1

Product URLs

http://www.oracle.com/technetwork/middleware/content-management/oit-all-085236.html

Details

In a PDF file an xref table contains multiple rows each containing three values ( except for the first row which specifies the first object being referenced and the number of objects). First value represents the 10 digit offset into the file where object is to be found. In a specially crafted PDF file, OID SDK PDF parser uses the specified value as a parameter in a call to realloc() which can fail. The return value is checked for errors but is subsequently ignored. The original numerical value is then used as an upper bound in a loop where out of bounds read happens during process cleanup.

In a function at 0xB74C603A in libvs_pdf.so (base address being 0xB74BF000) either a call to malloc or realloc is being made indirectly. Final size is calculated as follows:

`
.text:B74C608E mov     [edi+8], ecx
.text:B74C6091 imul    ecx, ebp
.text:B74C6094 mov     [esp+2Ch+var_14], ecx
.text:B74C6098 mov     [esp+2Ch+c], ecx
.text:B74C609C mov     eax, [edi]
.text:B74C609E mov     [esp+2Ch+s], eax
.text:B74C60A1 call    _SYSNativeReAlloc
`

Initially, register ecx holds the value from the file. In this case it is multiplied by 0x10 in ebp. The upper bound for the value in the file is 0x7ffffff, so the maximum size that can be passed to realloc is 0x7ffffff0. In limited memory conditions, of about less than 2 gigabytes of virtual memory available, this reallocation will fail returning zero. Although there are checks for this condition, the same buffer is iterated over during process cleanup. Specifically, in the following code in function VwStreamClose:

`
.text:B74D17C1                 add     esi, 10h 					[4]
.text:B74D17C4                 mov     eax, [esi] 					[1]
.text:B74D17C6                 test    eax, eax
.text:B74D17C8                 jz      short loc_B74D17D3
.text:B74D17CA                 mov     edx, [esp+4Ch+arg_4]
.text:B74D17CE                 call    sub_B74D14C0
.text:B74D17D3
.text:B74D17D3 loc_B74D17D3:                           ; CODE XREF: VwStreamClose+19Ej
.text:B74D17D3                 add     edi, 1 						[2]
.text:B74D17D6                 mov     eax, [esp+4Ch+arg_4]
.text:B74D17DA                 cmp     [eax+1D70h], edi 			[3]
.text:B74D17E0                 ja      short loc_B74D17C1
`

At [1], esi points to the buffer that was previously subject to failed realloc(). Register edi serves as a counter, is increased by 1 each turn at [2] and compared to the initial value from the file at [3]. At [4], pointer in esi is increased by 0x10. At this point, the parser expects that the memory reallocation was successful which leads to an out of bounds memory access.

The supplied minimized testcase triggers the out of bounds access in ixsample application supplied with the SDK. In order to trigger it, a virtual memory limit must be set by executing:

`
# ulimit -Sv 1000000
`

Timeline

2016-04-12 - Vendor Notification
2016-07-19 – Public Disclosure