There are two main parts to this assignment and each main part has a few problems. They make use of the files contained in this hw2 zip file. You should do this exercise using Microsoft's Visual C/C++ 6 compiler.

Part 1.

This part is about the "stack discipline", or how assembly language functions call each other. You should be familiar with the example code and references from Monday, January 26.

Problem 1.) In the "Part 1" sub folder of the hw2 zip file there is a C source file called mischief.c. For the first problem from this assignment, follow the instructions contained in that source code file. The main idea is that you are to write a function that modifies the return address stored in its stack frame so that the function returns somewhere other than it is supposed to (this is the main idea behind "buffer overflow attacks" or "smashing the stack").

Problem 2.) In the "Part 1" sub folder of hw2 zip file there is a C program called sittingDuck.c. Also in this folder is an executable sittingDuck.exe and a data file called exploit.bin. Do not compile the program. Use the executable given in the zip file. Run the executable with the data file as its input (using I/O redirection) by using the following command line

Problem 3.) The third problem of your assignment is to construct your own exploit2.bin input that, when read in by sittingDuck.exe, corrupts the stack of the function vulnerable() (in sittingDuck.c) in such a way that when vulnerable() returns, it returns to the call to the printf() function in the else part in the main() function (just like in the first problem of this assignment). Do not modify or recompile the program sittingDuck.c. Your exploit should work with the executable contained in the zip file. Test your exploit against the executable with a command line like the following:

The best strategy for doing the second and third problems of this assignment is to use the sittingDuck.asm assembly language listing and the disassembly window in the Visual Studio debugger to create a very detailed drawing of the running program's stack frames. (Such a drawing should be part of what you turn in for the second part.)

For more information about the use of the system stack for function calls, see these lecture notes from the operating systems course (15-410) at Carnegie Mellon University and these lecture notes from their archtecture course.

Here are some references about buffer overflows.

• Smashing The Stack For Fun And Profit
• Buffer Overflow Attacks and Their Countermeasures from Linux Journal
• A Look at the Buffer-Overflow Hack from Linux Journal
• Buffer Overflows for Dummies from SANS InfoSec Reading Room
• Buffer Overflow - Wikipedia
• Slammed! An inside view of the worm that crashed the Internet in 15 minutes. from Wired magazine

Part 2.

This part is about calling conventions, linking, and dll's. You should be familiar with the example code and references from Wednesday, January 28. There are two problems in this part. They make use of the files contained in the "Part 2" sub folder of the hw2 zip file. For these problems you will need to download several utility programs.

• depends21_x86.zip from the Dependency Walker home page.
• PrcView_5_2_15.zip from the Process Viewer home page.
• PEBrowse.zip from the PEBrowse Professional home page.

Problem 4.) The first problem from this part asks you to answer a few questions about dll's, just to get you more familiar with them. Write up an answer to each of the following questions.

4a.) How many dll's are there on the system drive of your computer? (Your system drive is the environment variable value %SystemDrive%.) Explain how you figured out your answer. How many dll's are there in your %SystemRoot%?

4b.) For this question, you need Dependency Walker. This program is used to examine part of the structure of exe and dll files. It tells you all of the dll's that an exe (or dll) file will "load" when the file executes. It also tells you which functions an exe imports from each dll, and it tells you all of the functions that a dll exports. Use Dependency Walker to open notepad.exe. How many dll's does notepad.exe use? (Hint: Click on the name of one dll in the lower window (the "Module List View"), use the "Edit -> Select All" menu item, then use copy and paste to get all of the dll names into an editor window, and then see how many lines there are.) Does the notepad.exe program make use of the C Library printf() function? (C Library functions are contained in MSVCRT.DLL.)

4c.) In the "Part 2" sub folder of the hw2 zip file there is a copy of a program called M.exe (downloaded from www.magma.ca/~wjr/). Notice the very small size of M.exe. How many dll's does M.exe make use of? Now run M.exe by double clicking on it, but do not do anything with the running program yet. Use the program PrcView.exe to find out how many dll's have been loaded into memory by the running instance of M.exe. Now go back to the running instance of M.exe and click on the "File -> Save As ..." menu item (but do not do anything yet with the dialogue box that opens up). Now how many dll's does the running instance of M.exe have loaded into memory? (You may need to refresh the relevant window of PrcView.exe. Click on the window and tap the F5 key.) Play around with M.exe. It is a nice little program. It is entirely written in assembly language (and there is a newer version available from its home page). Try loading the data file nice.m. Then click on the menu item "Colors -> Add Shifting" and watch the animation (it runs for quite a while).

4c'.) If you wish, you can also redo the previous problem using either of the two programs LoadLibrarySpy.exe or ProcessSpy.exe found in the hw2 zip file. These two programs come from the MSDN article Escape from DLL Hell with Custom Debugging and Instrumentation Tools and Utilities.

4d.) In the previous two exercises, you looked at a program and examined which dll's it used. In this exercise, you will look at a dll and find out which processes are currently (and simultaneously) using it. Start the program DllSpy.exe found in the "Part 2" subfolder of the hw2 zip file (this program also comes from the article Escape from DLL Hell with Custom Debugging and Instrumentation Tools and Utilities). Use it to find out how many processes currently running on your system have the dll msvcrt.dll loaded into their memory space (this dll is the C Library). Also see how many processes currently running on your system have the dll wsock32.dll loaded into their memory space (this dll contains functions for connecting to the Internet). Use DllSpy.exe to count how many dll's are currently loaded into your computer's memory. Explain how you get your count.

Problem 5.) In the "Part 2" sub folder of the hw2 zip file there is an exe file called dllBlues.exe. When you run this program, it has a problem. Your assignment is to solve all of its problems so that the program can run properly (dllBlues.exe is not supposed to really do anything specific, so your job is to just provide it with whatever it needs so that it runs without generating any errors). You will need to examine what is inside of dllBlues.exe so that you can figure out what you need to do. Use Dependency Walker and PEBrowse Professional as your examining tools.

PEBrowse Professional is a dissasembler. To get started with it, run PEBrowsePro.exe and use the "File -> Open File..." menu item to open dllBlues.exe (or just use "drag and drop" and drop the dllBlues.exe icon onto the PEBrowsePro.exe icon). Then click on the little "+" symbol next to the word "Sections" in the left hand pane. Then right click on the item ".text" and choose the "Disassemble .text" menu item (you might be prompted for the source code to dllBlues.c, but you don't have that, so click on Cancel). You will then see a disassembly window for dllBlues.exe. There is a lot more that PEBrowsePro.exe can tell you about dllBlues.exe, so do some exploring.

There are other disassemblers that you can try to use on dllBlues.exe. For example, there is objdump.exe that comes with MinGW.

Here are a few references to some very useful documentation about the Microsoft compiler and linker from Visual Studio 6.

• Compiling and Linking - MSDN
• Compiler Command Line Reference _ MSDN
• Linker Command Line Reference - MSDN
• Compiler-Controlled LINK Options - MSDN
• Dynamic-Link Library Creation - MSDN

Turn in a zip file containing your version of mischief.c from the first problem of this assignment, your explanation of the second problem, and your exploit2.bin file for the third problem. If you want to know if your explanation is on track and sufficiently detailed, send me a copy before the due date and I'll let you know how you are doing.

For the second part of this assignment you will end up writing a dll. Turn in (in your zip file) your dll's source code, your def file for your dll, your compiled and linked dll and lib files, and your written answers to the questions from Problem 4.

This assignment is due Monday, February 9.