Really weird use after free in HumanBehaviors.lua?? #230
Description
I got bored and decided to compile Cortex Command with gcc's -fsanitize=address enabled. I will never get back the last 4 hours of my life.
With it, the game initially runs fine, but on enemy engagement the sanitizer detects a use-after-free error and crashes quite reliably. Here's its summary:
Address sanitizer report
=================================================================
==129375==ERROR: AddressSanitizer: heap-use-after-free on address 0x7ba4e92090bc at pc 0x555ceea838ce bp 0x7b84d1cd58d0 sp 0x7b84d1cd58c0
READ of size 4 at 0x7ba4e92090bc thread T40
#0 0x555ceea838cd in RTE::operator+(RTE::Vector const&, RTE::Vector const&) ../Source/System/Vector.h:392
#1 0x555cf0ece824 in luabind::operators::add::apply<RTE::Vector const&, RTE::Vector const&, luabind::detail::null_type>::execute(lua_State*, RTE::Vector const&, RTE::Vector const&) ../external/sources/luabind-0.7.1/luabind/operator.hpp:282
#2 0x555cf0eb7af7 in luabind::detail::binary_operator<luabind::operators::add, luabind::const_self_type, luabind::other<RTE::Vector const&> >::apply<RTE::Vector, luabind::detail::null_type>::execute(lua_State*, RTE::Vector const&, RTE::Vector const&) ../external/sources/luabind-0.7.1/luabind/operator.hpp:140
#3 0x555cf0f6ba48 in int luabind::detail::returns<void>::call<RTE::Vector, luabind::detail::policy_cons<luabind::detail::raw_policy<1>, luabind::detail::null_type>, lua_State*, RTE::Vector const&, RTE::Vector const&>(void (*)(lua_State*, RTE::Vector const&, RTE::Vector const&), RTE::Vector*, lua_State*, luabind::detail::policy_cons<luabind::detail::raw_policy<1>, luabind::detail::null_type> const*) ../external/sources/luabind-0.7.1/luabind/detail/call.hpp:353
#4 0x555cf0f5bd03 in int luabind::detail::call<RTE::Vector, luabind::detail::policy_cons<luabind::detail::raw_policy<1>, luabind::detail::null_type>, void, lua_State*, RTE::Vector const&, RTE::Vector const&>(void (*)(lua_State*, RTE::Vector const&, RTE::Vector const&), RTE::Vector*, lua_State*, luabind::detail::policy_cons<luabind::detail::raw_policy<1>, luabind::detail::null_type> const*) ../external/sources/luabind-0.7.1/luabind/detail/call.hpp:386
#5 0x555cf0f395c9 in luabind::detail::mem_fn_callback<void (*)(lua_State*, RTE::Vector const&, RTE::Vector const&), RTE::Vector, luabind::detail::policy_cons<luabind::detail::raw_policy<1>, luabind::detail::null_type> >::operator()(lua_State*) const ../external/sources/luabind-0.7.1/luabind/class.hpp:223
#6 0x555cf0f212fd in boost::detail::function::function_obj_invoker1<luabind::detail::mem_fn_callback<void (*)(lua_State*, RTE::Vector const&, RTE::Vector const&), RTE::Vector, luabind::detail::policy_cons<luabind::detail::raw_policy<1>, luabind::detail::null_type> >, int, lua_State*>::invoke(boost::detail::function::function_buffer&, lua_State*) ../external/include/boost_1_75/boost/function/function_template.hpp:137
#7 0x555ceff84b7f in boost::function1<int, lua_State*>::operator()(lua_State*) const ../external/include/boost_1_75/boost/function/function_template.hpp:763
#8 0x555ceff85b70 in luabind::detail::overload_rep::call(lua_State*, bool) const ../external/sources/luabind-0.7.1/src/overload_rep.cpp:32
#9 0x555ceff76472 in luabind::detail::class_rep::function_dispatcher(lua_State*) ../external/sources/luabind-0.7.1/src/class_rep.cpp:667
#10 0x555cefef1575 in lj_BC_FUNCC external/sources/LuaJIT-2.1/src/lj_vm.s:857
#11 0x555cefed3fdf in lua_call ../external/sources/LuaJIT-2.1/src/lj_api.c:1127
#12 0x555ceff75250 in luabind::detail::class_rep::operator_dispatcher(lua_State*) ../external/sources/luabind-0.7.1/src/class_rep.cpp:404
#13 0x555cefef1575 in lj_BC_FUNCC external/sources/LuaJIT-2.1/src/lj_vm.s:857
#14 0x555cefef23ab in lj_ff_coroutine_resume external/sources/LuaJIT-2.1/src/lj_vm.s:1740
#15 0x555cefed4083 in lua_pcall ../external/sources/LuaJIT-2.1/src/lj_api.c:1145
#16 0x555ceffc5fa7 in RTE::LuaStateWrapper::RunScriptFunctionObject(RTE::LuabindObjectWrapper const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<RTE::Entity const*, std::allocator<RTE::Entity const*> > const&, std::vector<std::basic_string_view<char, std::char_traits<char> >, std::allocator<std::basic_string_view<char, std::char_traits<char> > > > const&, std::vector<RTE::LuabindObjectWrapper*, std::allocator<RTE::LuabindObjectWrapper*> > const&) ../Source/Managers/LuaMan.cpp:634
#17 0x555ceee1df6b in RTE::MovableObject::RunScriptedFunctionInAppropriateScripts(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool, std::vector<RTE::Entity const*, std::allocator<RTE::Entity const*> > const&, std::vector<std::basic_string_view<char, std::char_traits<char> >, std::allocator<std::basic_string_view<char, std::char_traits<char> > > > const&, std::vector<RTE::LuabindObjectWrapper*, std::allocator<RTE::LuabindObjectWrapper*> > const&) ../Source/Entities/MovableObject.cpp:674
#18 0x555cef45d008 in operator() ../Source/Managers/MovableMan.cpp:1780
#19 0x555cef476f07 in __invoke_impl<void, const RTE::MovableMan::UpdateControllers()::<lambda(int, int)>&, long unsigned int const&, long unsigned int const&> /usr/include/c++/15.2.1/bits/invoke.h:63
#20 0x555cef476a2e in __invoke<const RTE::MovableMan::UpdateControllers()::<lambda(int, int)>&, long unsigned int const&, long unsigned int const&> /usr/include/c++/15.2.1/bits/invoke.h:98
#21 0x555cef476270 in __call_c<void, 0, 1> /usr/include/c++/15.2.1/functional:526
#22 0x555cef4759ec in operator()<> /usr/include/c++/15.2.1/functional:612
#23 0x555cef474e9f in __invoke_impl<void, const std::_Bind<RTE::MovableMan::UpdateControllers()::<lambda(int, int)>(long unsigned int, long unsigned int)>&> /usr/include/c++/15.2.1/bits/invoke.h:63
#24 0x555cef4746f4 in __invoke<const std::_Bind<RTE::MovableMan::UpdateControllers()::<lambda(int, int)>(long unsigned int, long unsigned int)>&> /usr/include/c++/15.2.1/bits/invoke.h:98
#25 0x555cef473aa2 in invoke<const std::_Bind<RTE::MovableMan::UpdateControllers()::<lambda(int, int)>(long unsigned int, long unsigned int)>&> /usr/include/c++/15.2.1/functional:122
#26 0x555cef472db5 in operator() ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:486
#27 0x555cef476ff9 in __invoke_impl<void, BS::thread_pool::submit<RTE::MovableMan::UpdateControllers()::<lambda(int, int)>, long unsigned int, long unsigned int>(RTE::MovableMan::UpdateControllers()::<lambda(int, int)>&&, long unsigned int&&, long unsigned int&&)::<lambda()>&> /usr/include/c++/15.2.1/bits/invoke.h:63
#28 0x555cef476f90 in __invoke<BS::thread_pool::submit<RTE::MovableMan::UpdateControllers()::<lambda(int, int)>, long unsigned int, long unsigned int>(RTE::MovableMan::UpdateControllers()::<lambda(int, int)>&&, long unsigned int&&, long unsigned int&&)::<lambda()>&> /usr/include/c++/15.2.1/bits/invoke.h:98
#29 0x555cef476f27 in __call<void> /usr/include/c++/15.2.1/functional:515
#30 0x555cef476ad6 in operator()<> /usr/include/c++/15.2.1/functional:600
#31 0x555cef47631d in __invoke_impl<void, std::_Bind<BS::thread_pool::submit<RTE::MovableMan::UpdateControllers()::<lambda(int, int)>, long unsigned int, long unsigned int>(RTE::MovableMan::UpdateControllers()::<lambda(int, int)>&&, long unsigned int&&, long unsigned int&&)::<lambda()>()>&> /usr/include/c++/15.2.1/bits/invoke.h:63
#32 0x555cef475b4d in __invoke_r<void, std::_Bind<BS::thread_pool::submit<RTE::MovableMan::UpdateControllers()::<lambda(int, int)>, long unsigned int, long unsigned int>(RTE::MovableMan::UpdateControllers()::<lambda(int, int)>&&, long unsigned int&&, long unsigned int&&)::<lambda()>()>&> /usr/include/c++/15.2.1/bits/invoke.h:113
#33 0x555cef474ff8 in _M_invoke /usr/include/c++/15.2.1/bits/std_function.h:292
#34 0x555ceee73e99 in std::function<void ()>::operator()() const /usr/include/c++/15.2.1/bits/std_function.h:593
#35 0x555cef570ed3 in BS::thread_pool::worker() ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:637
#36 0x555cef574462 in void std::__invoke_impl<void, void (BS::thread_pool::*)(), BS::thread_pool*>(std::__invoke_memfun_deref, void (BS::thread_pool::*&&)(), BS::thread_pool*&&) /usr/include/c++/15.2.1/bits/invoke.h:76
#37 0x555cef574350 in std::__invoke_result<void (BS::thread_pool::*)(), BS::thread_pool*>::type std::__invoke<void (BS::thread_pool::*)(), BS::thread_pool*>(void (BS::thread_pool::*&&)(), BS::thread_pool*&&) /usr/include/c++/15.2.1/bits/invoke.h:98
#38 0x555cef5742d2 in void std::thread::_Invoker<std::tuple<void (BS::thread_pool::*)(), BS::thread_pool*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/include/c++/15.2.1/bits/std_thread.h:303
#39 0x555cef57428b in std::thread::_Invoker<std::tuple<void (BS::thread_pool::*)(), BS::thread_pool*> >::operator()() /usr/include/c++/15.2.1/bits/std_thread.h:310
#40 0x555cef57426f in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (BS::thread_pool::*)(), BS::thread_pool*> > >::_M_run() /usr/include/c++/15.2.1/bits/std_thread.h:255
#41 0x7f84ec1002f3 in execute_native_thread_routine /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/thread.cc:104
#42 0x7f84ed065ca1 in asan_thread_start /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_interceptors.cpp:239
#43 0x7f84ebca80d5 (/usr/lib/libc.so.6+0xa80d5) (BuildId: 3fb5bf3586fec17ba65a16ec9a3132455897d306)
#44 0x7f84ebd3b01b (/usr/lib/libc.so.6+0x13b01b) (BuildId: 3fb5bf3586fec17ba65a16ec9a3132455897d306)
0x7ba4e92090bc is located 12 bytes inside of 16-byte region [0x7ba4e92090b0,0x7ba4e92090c0)
freed by thread T43 here:
#0 0x7f84ed154e8d in operator delete(void*, unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:155
#1 0x555ceea60992 in RTE::Vector::~Vector() ../Source/System/Vector.h:16
#2 0x555cf0eea33d in luabind::detail::delete_s<RTE::Vector>::apply(void*) ../external/sources/luabind-0.7.1/luabind/detail/object_rep.hpp:93
#3 0x555ceff857ec in luabind::detail::object_rep::~object_rep() ../external/sources/luabind-0.7.1/src/object_rep.cpp:53
#4 0x555ceff85a30 in luabind::detail::object_rep::garbage_collector(lua_State*) ../external/sources/luabind-0.7.1/src/object_rep.cpp:107
#5 0x555cefef1575 in lj_BC_FUNCC external/sources/LuaJIT-2.1/src/lj_vm.s:857
previously allocated by thread T48 here:
#0 0x7f84ed153d2d in operator new(unsigned long) /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_new_delete.cpp:86
#1 0x555cf0eea50a in RTE::Vector* luabind::detail::constructor_helper<2>::execute<RTE::Vector, luabind::detail::null_type, float, float, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type>(lua_State*, luabind::weak_ref const&, RTE::Vector*, luabind::constructor<float, float, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type>*, luabind::detail::null_type*) ../external/sources/luabind-0.7.1/luabind/detail/constructor.hpp:122
#2 0x555cf0ece0a8 in luabind::detail::construct_class<RTE::Vector, luabind::detail::null_type, luabind::constructor<float, float, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type, luabind::detail::null_type> >::apply(lua_State*, luabind::weak_ref const&) ../external/sources/luabind-0.7.1/luabind/detail/constructor.hpp:65
#3 0x555ceff7a4e1 in luabind::detail::construct_rep::overload_t::construct(lua_State*, luabind::weak_ref const&) ../external/sources/luabind-0.7.1/luabind/detail/construct_rep.hpp:61
#4 0x555ceff75879 in luabind::detail::class_rep::constructor_dispatcher(lua_State*) ../external/sources/luabind-0.7.1/src/class_rep.cpp:497
#5 0x555cefef1575 in lj_BC_FUNCC external/sources/LuaJIT-2.1/src/lj_vm.s:857
Thread T40 created by T0 here:
#0 0x7f84ed147670 in pthread_create /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_interceptors.cpp:250
#1 0x7f84ec1003b9 in __gthread_create(unsigned long*, void* (*)(void*), void*) /usr/src/debug/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:709
#2 0x7f84ec1003b9 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/thread.cc:172
#3 0x555cef570851 in BS::thread_pool::create_threads() ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:580
#4 0x555cef570568 in BS::thread_pool::reset(unsigned int) ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:462
#5 0x555cef56fe49 in RTE::ThreadMan::Clear() ../Source/Managers/ThreadMan.cpp:15
#6 0x555cef56fd43 in RTE::ThreadMan::ThreadMan() ../Source/Managers/ThreadMan.cpp:6
#7 0x555ceea4a352 in RTE::Singleton<RTE::ThreadMan>::Construct() ../Source/System/Singleton.h:25
#8 0x555ceea3e8c7 in InitializeManagers() ../Source/Main.cpp:75
#9 0x555ceea41cf8 in main ../Source/Main.cpp:442
#10 0x7f84ebc27b8a (/usr/lib/libc.so.6+0x27b8a) (BuildId: 3fb5bf3586fec17ba65a16ec9a3132455897d306)
Thread T43 created by T0 here:
#0 0x7f84ed147670 in pthread_create /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_interceptors.cpp:250
#1 0x7f84ec1003b9 in __gthread_create(unsigned long*, void* (*)(void*), void*) /usr/src/debug/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:709
#2 0x7f84ec1003b9 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/thread.cc:172
#3 0x555cef570851 in BS::thread_pool::create_threads() ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:580
#4 0x555cef570568 in BS::thread_pool::reset(unsigned int) ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:462
#5 0x555cef56fe49 in RTE::ThreadMan::Clear() ../Source/Managers/ThreadMan.cpp:15
#6 0x555cef56fd43 in RTE::ThreadMan::ThreadMan() ../Source/Managers/ThreadMan.cpp:6
#7 0x555ceea4a352 in RTE::Singleton<RTE::ThreadMan>::Construct() ../Source/System/Singleton.h:25
#8 0x555ceea3e8c7 in InitializeManagers() ../Source/Main.cpp:75
#9 0x555ceea41cf8 in main ../Source/Main.cpp:442
#10 0x7f84ebc27b8a (/usr/lib/libc.so.6+0x27b8a) (BuildId: 3fb5bf3586fec17ba65a16ec9a3132455897d306)
Thread T48 created by T0 here:
#0 0x7f84ed147670 in pthread_create /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_interceptors.cpp:250
#1 0x7f84ec1003b9 in __gthread_create(unsigned long*, void* (*)(void*), void*) /usr/src/debug/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:709
#2 0x7f84ec1003b9 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/thread.cc:172
#3 0x555cef570851 in BS::thread_pool::create_threads() ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:580
#4 0x555cef570568 in BS::thread_pool::reset(unsigned int) ../external/include/thread-pool-3.5.0/include/BS_thread_pool.hpp:462
#5 0x555cef56fe49 in RTE::ThreadMan::Clear() ../Source/Managers/ThreadMan.cpp:15
#6 0x555cef56fd43 in RTE::ThreadMan::ThreadMan() ../Source/Managers/ThreadMan.cpp:6
#7 0x555ceea4a352 in RTE::Singleton<RTE::ThreadMan>::Construct() ../Source/System/Singleton.h:25
#8 0x555ceea3e8c7 in InitializeManagers() ../Source/Main.cpp:75
#9 0x555ceea41cf8 in main ../Source/Main.cpp:442
#10 0x7f84ebc27b8a (/usr/lib/libc.so.6+0x27b8a) (BuildId: 3fb5bf3586fec17ba65a16ec9a3132455897d306)
SUMMARY: AddressSanitizer: heap-use-after-free ../Source/System/Vector.h:392 in RTE::operator+(RTE::Vector const&, RTE::Vector const&)
Shadow bytes around the buggy address:
0x7ba4e9208e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7ba4e9208e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7ba4e9208f00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7ba4e9208f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7ba4e9209000: fa fa fa fa fa fa fa fa fa fa fd fd fa fa fd fd
=>0x7ba4e9209080: fa fa fd fd fa fa fd[fd]fa fa fd fd fa fa fd fa
0x7ba4e9209100: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7ba4e9209180: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7ba4e9209200: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7ba4e9209280: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x7ba4e9209300: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==129375==ABORTING
It's a huge mess which doesn't tell me much, but it does tell that it's something in Lua stuff, and the free happens in the garbage collection stage.
After lots and lots of careful deliberation (killing override of Lua print and adding print statements literally everywhere) I have found it to be in HumanBehaviors.ShootTarget, and specifically after a yield.
Here's the line with yield:
If I print(tostring(ErrorOffset)) right before this yield, it prints fine then yields. If I print after this line, then comes the sanitizer error. In this function's execution flow, somehow, this yield point causes ErrorOffset to be thrown away.
LuaJIT bug? Luabind bug?? Some fuckup of Lua state in a way that looks completely legal to the sanitizer right up until this point?? I really don't know, and I don't know if I have the mental capacity to debug further.
At least it's not a multithreading problem (I THINK). If I set NumberOfLuaStatesOverride = 1 in Settings.ini, and even if I also edit ThreadMan to spawn only 1 thread in each pool, it still happens (but the prints are not interleaved anymore lmao)
Also, if I do this:
diff --git a/Data/Base.rte/AI/HumanBehaviors.lua b/Data/Base.rte/AI/HumanBehaviors.lua
index 57a41815b..05d25b0fa 100644
--- a/Data/Base.rte/AI/HumanBehaviors.lua
+++ b/Data/Base.rte/AI/HumanBehaviors.lua
@@ -1290,7 +1290,9 @@ function HumanBehaviors.ShootTarget(AI, Owner, Abort)
Owner.HFlipped = true;
end
+ local amongus = Vector(ErrorOffset.X, ErrorOffset.Y);
local _ai, _ownr, _abrt = coroutine.yield(); -- wait until next frame
+ ErrorOffset = amongus;
if _abrt then return true end
endIt no longer detects any errors, and I can easily complete the tutorial level with no issue. There might be more memory sanitizer errors I haven't run into yet. My neck hurts and I am tired and I really need sleep. That was fun!