Talos Vulnerability Report

DESCRIPTION

A stack overflow leading to a crash due to unbounded recusive function call is present in the PDF file format parsing code of the IX SDK.

TESTED VERSIONS

Oracle Outside In IX sdk 8.5.1

PRODUCT URLs

http://www.oracle.com/technetwork/middleware/content-management/oit-all-085236.html

DETAILS

While parsing a malformed PDF file which contains a reference to the Root element with malformed or missing an xref table a recursive call to a function is made each time with the same parameters eventualy leading to a crash due to process stack exhaustion.

Technical information below:

During a call to VwStreamOpen function in libvs_pdf.so library, code dealing with Root element is reached (image base is at 0xB74BF000):

.text:B74ED100 loc_B74ED100:
.text:B74ED100 lea     ebp, [esp+6BCh+var_BC]
.text:B74ED107 cld
.text:B74ED108 mov     ecx, 8
.text:B74ED10D xor     eax, eax
.text:B74ED10F mov     edi, ebp
.text:B74ED111 rep stosd
.text:B74ED113 lea     ecx, [esp+6BCh+var_34]
.text:B74ED11A mov     eax, [esp+6BCh+arg_10]
.text:B74ED121 mov     [esp+6BCh+s], eax
.text:B74ED124 lea     edx, (aRoot - 0B74F6998h)[ebx] ; "Root"
.text:B74ED12A mov     eax, esi
.text:B74ED12C call    sub_B74D653E
.text:B74ED131 mov     edx, eax
.text:B74ED133 test    ax, ax
.text:B74ED136 jnz     loc_B74E

Function sub_B74D653E in turn calls a function sub_B74D5EEC in which the unbounded recursive call can happen:

.text:B74D6095 lea     edx, [esp+5ACh+var_14]
.text:B74D609C lea     eax, [esp+5ACh+var_C0]
.text:B74D60A3 mov     ecx, ebp
.text:B74D60A5 call    sub_B74D5EEC
.text:B74D60AA test    ax, ax
.text:B74D60AD jnz     short loc_B74

The supplied minimized testcase triggers the recursive call and leads to a crash due to stack exhaustion. The sample program ixsample supplied with the SDK can be used to reproduce the crash.

TIMELINE

2016-04-12 - Vendor Notification
2016-07-19 - Public Disclosure