Beneath the Waves

Time 2021-11-16 08:14:23

Web Name: Beneath the Waves

WebSite: http://www.beneaththewaves.net

ID:234625

Keywords:

Beneath,the,Waves,version,code,Added,Posted,data,source,additional,Once,will,fil

Description:

keywords:version, code, Added, Posted, data, source, additional, Once, will, file, Remembers, Dust, CreateSkeletonexe, SymDumpTEexe, flag, original, Ghidra, files, release, global
description:Beneath the Waves

Home

This is the personal website of Ben Lincoln.

Messenger Bag Display

For Def Con 27 this past summer, I put together a Raspberry Pi Messenger Bag Display, and I finally collected some photos, video, and the code I wrote, if anyone is curious.

I'm still working on the next release of This Dust Remembers What It Once Was, but expect to see that soon.

If you're a fan of Soul Reaver, check out a sneak preview of a massive amount of deleted content that will be revealed next year, over at my other site, Legacy of Kain: The Lost Worlds.

Posted by Ben Lincoln, 2019-11-17 @ 11:33 This Dust Remembers What It Once Was version 0.9

This Dust Remembers What It Once Was version 0.9 has been released. The most significant addition in this release is the --inline-fakes option for SymDumpTE.exe. It will move nearly all of the "fake" structs and unions inline into other structs/unions/etc. where they actually belong. There are some other improvements as well. I've not had time to test this as thoroughly as previous versions, so it may be a bit buggy with titles other than Soul Reaver. If you get stuck in the meantime, using 0.8 is a good fallback.

Posted by Ben Lincoln, 2019-09-25 @ 07:00 This Dust Remembers What It Once Was version 0.8

This Dust Remembers What It Once Was version 0.8 has been released. This is the biggest update since the first release, and includes the following changes:

Added support for PsyQ overlays to PlayStationELFConverter.exe, SymDumpTE.exe, CreateSkeleton.exe, and PopulateSkeleton.exe. Added a global variable/data-exporting Ghidra script. This is a data counterpart to the existing decompiled-code-exporting script, and outputs global variables/embedded data as C code. This should save an enormous amount of time when attempting to reconstruct source code. Added a custom array-detection Ghidra script. This automatically detects most global arrays and redefines them as such in Ghidra. Combined with the data-exporting script, a huge chunk of manual reverse-engineering effort can generally be avoided. Added fully-functional handlers for most remaining SYM file entry types to SymDumpTE.exe, and stub handlers for the rest. This provides a significant amount of additional data about PsyQ-based PlayStation binaries. Greatly-improved handling of labels, as the label type is now identified. Labels of type 1 (which generally are relative offsets, memory segment boundaries, or other data that should NOT be treated as function/variable names) are now filtered out from inclusion in the Ghidra scripts where applicable. Multiple additional sources of forensic information regarding the original source and/or object files are now parsed from the SYM data and used by TDR to map functions and data to their proper source files. Added an --auto-rename-fakes flag to SymDumpTE.exe. This automatically renames "fake" structs and unions to avoid name conflicts and generate more accurate results. Using this option is highly recommended. Improved handling of 'fake' structs and unions even if --auto-rename-fakes is not used. Added an --output-updated-json option to CreateSkeleton.exe which will create an updated version of the JSON debug symbol file. This updated version will contain additional data either generated by CreateSkeleton.exe, or forensically determined from the newly-analyzed additional symbol data, and can be passed to PopulateSkeleton.exe instead of the original JSON file for a more accurate reconstruction. Optionally, the file can be manually edited based on user knowledge/intentions so that PopulateSkeleton.exe will place certain code elements in locations of the user's choice even if they could not be automatically determined by TDR. Added a --map-sld-functions flag to CreateSkeleton.exe. This uses some of the previously-ignored data in the symbol file to map functions without explicit function definitions (library calls, etc.) to original source code files. Using this flag is highly recommended as it should greatly reduce the number of unmapped functions in the decompiled code output. Added a --use-gp-base option to CreateSkeleton.exe. Use this to manually specify a global pointer value which should be assumed at the beginning of all functions in Ghidra. This works like --assume-sn-gp-base, but requires that the global pointer value be specified by the user. This option is intended for use with games like Diablo which do not include a __SN_GP_BASE value in their debug symbols, and requires manual effort to determine. Added a --ignore-duplicate-definitions flag to SymDumpTE.exe. This will suppress messages about duplicate (identical) definitions of structs, enums, etc. Messages about redefinitions/name conflicts will still be output even when this flag is used. When creating stub source code and decompiled code, CreateSkeleton.exe and PopulateSkeleton.exe now include two additional directories in the path to the output files. The first is named PRIMARY for the standard executable code, and after the overlay ID for any additional overlay binaries handled by TDR. The second named after the base drive letter where the original source code was located, in order to avoid conflicts in cases where the original developer had identical paths with different files on different drives.
For example, a file generated by TDR 0.7 and earlier might be located at source-stubs/kain2/game/RAZIEL/RAZIEL.C. In version 0.8 and later, that same file would be located at source-stubs/PRIMARY/C/kain2/game/RAZIEL/RAZIEL.C. Added a --replace-non-ascii-labels flag to CreateSkeleton.exe as an alternative to the more blunt --ignore-non-ascii-labels. Corrected some minor memory segment naming issues. SymDumpTE.exe and PopulateSkeleton.exe now correctly display their help messages. Added some input validation to help prevent invalid Java script content in the event of unusual data, or malicious code execution in the event that malicious, crafted PlayStation binaries are processed. Externs are now named correctly in the JSON files. Additional work on general code cleanup to make the source less horrific (still in progress). Reducing the overall horror did lead to some localized increases in horror.

In short, it supports a bunch of things it didn't before, works much better, and requires significantly less manual work by the user.

I added a very lengthy, detailed tutorial which introduces the overlay-handling features using another custom PSX EXE: TDR: Practice Using OVERLAYS. I recommend going through that walkthrough even if you're not interested in games that use memory overlays, because it has a bunch of additional information about hwo to use TDR. It also has some comparisons between original source code and decompiled versions.

TDR: Practice Using EDGECASE has been updated with a revised version of that binary and some additional steps.

TDR: Soul Reaver, TDR: Need For Speed 4, and TDR: Biohazard 2 have all been revised significantly because most of the manual workarounds necessary with older versions of TDR are no longer required.

Posted by Ben Lincoln, 2019-09-11 @ 05:00 This Dust Remembers What It Once Was version 0.7

The 0.7 release of This Dust Remembers What It Once Was is ready. This one once again significantly improves results, and I fixed it to work with Biohazard 2 again.

Posted by Ben Lincoln, 2019-08-22 @ 00:30 This Dust Remembers What It Once Was version 0.6

Another alpha release (0.6) of This Dust Remembers What It Once Was is ready. This one should work even better than previous releases.

I also added a much more basic walkthrough of the tool, with a custom toy PsyQ PlayStation binary which has source included for comparison. You can find that at TDR: Practice Using EDGECASE.

Posted by Ben Lincoln, 2019-08-19 @ 18:00 This Dust Remembers What It Once Was version 0.4

Just released a third alpha version (0.4) of This Dust Remembers What It Once Was. This one should work much better than previous releases.

I also finishd the walkthrough for using it to decompile a beta version of Biohazard 2, which you can find in TDR: Biohazard 2.

Posted by Ben Lincoln, 2019-08-14 @ 23:20 This Dust Remembers What It Once Was version 0.3

Today I'm releasing a second alpha version of This Dust Remembers What It Once Was. This one has some major improvements and new capabilities.

I recently discovered that mail sent to me through the Contact forms on my websites has been frequently going nowhere, because my hosting provider doesn't think outbound email from web forms and other similar systems should be reliable. If you've tried to get in touch with me and haven't seen a response, I've added my social media contact information to the Contact page.

Posted by Ben Lincoln, 2019-08-13 @ 23:50 This Dust Remembers What It Once Was

Today I'm releasing a very early alpha version of This Dust Remembers What It Once Was, a videogame reverse-engineering toolkit for use with Ghidra. This initial version is focused almost exclusively on PlayStation games.

If you scanned my QR code, the details are over at Legacy of Kain: The Lost Worlds, but feel free to contact me via either site.

Posted by Ben Lincoln, 2019-08-06 @ 07:00 Summon the Lulz

Summon the Lulz.

Posted by Ben Lincoln, 2018-01-06 @ 15:00 Assorted New Material

In this completely random collection of updates, I present:

A long-overdue rewrite of Thermal versus Near Infrared, prompted by a discussion I had in email with Dr. David Wilson.

A look at how I made my Destiny Costume for Halloween, 2017.

A list of the vulnerabilities I've discovered which have been publicly-disclosed: Hack the Planet.

Posted by Ben Lincoln, 2018-01-02 @ 08:30

Archived Update Posts: 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2018 | 2019

Last updated: 13 August 2019

You have Javascript disabled in your browser. This website will look a bit nicer if you turn it back on.

Copyright 2009-2019 Ben Lincoln, except where explicitly noted.

Copyright Information - Privacy Policy - Download Policy - Advertising Policy

TAGS:Beneath the Waves version code Added Posted data source additional Once will fil

<<< Thank you for your visit >>>

Beneath the Waves

Hot Websites