.NET LLaMA.cpp library! 🦙 #494

hpretila · 2023-03-25T12:25:56Z

hpretila
Mar 25, 2023

Got bored today and slapped together a .NET Core library wrapped around llama.cpp! Got inference working, but still need to work through the model conversions 😁

using LLaMA.NET;

LLaMAModel model = LLaMAModel.FromPath("/path/to/7B-LoRA/ggml-model-q4_0.bin");
LLaMARunner runner = model.CreateRunner()

var res = runner
    .WithPrompt(" This is the story of a man named ")
    .Infer(out _, nTokensToPredict = 50);

Console.Write(res);

model.Dispose();

afsanchez001 · 2023-03-31T04:24:54Z

afsanchez001
Mar 31, 2023

3 replies

hpretila Mar 31, 2023
Author

Can you try building and preparing Llama.cpp with python scripts/build_llama_cpp.py?

Edit: Ensure you've pulled the llama.cpp submodule at /llama.cpp/., and that you have Visual Studio installed with C/C++ tools on Windows, or gcc on linux.

afsanchez001 Mar 31, 2023

Hi hpretila, thank you so much for getting back to me. I was so ( ( frustrated ) ) with myself last night that I deleted my comments and went to bed. lol

Okay, this is my solution folder: C:\Users\PC\Desktop\llama.net-main

llama.cpp <-- in here is the entire llama.cpp submodule
LLaMA.NET
LLaMA.NET.Testing

I do have both cmake version 3.26.2 and Python 3.11.2 installed.

I then did this:

cd C:\Users\PC\Desktop\llama.net-main
python scripts/build_llama_cpp.py

And this was the output:

-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22000.0 to target Windows 10.0.19044.
-- The C compiler identification is MSVC 19.34.31937.0
-- The CXX compiler identification is MSVC 19.34.31937.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: D:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: D:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- x86 detected
-- Configuring done (6.1s)
-- Generating done (0.3s)
-- Build files have been written to: C:/Users/PC/Desktop/llama.net-main/llama.cpp/build
MSBuild version 17.4.1+9a89d02ff for .NET Framework
Checking Build System
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/CMakeLists.txt
ggml.c
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(730,33): warning C4244: 'initializing': conversion from 'float' to
'const uint8_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(731,33): warning C4244: 'initializing': conversion from 'float' to
'const uint8_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(1717,10): warning C4244: '=': conversion from 'ggml_float' to 'floa
t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(2000,10): warning C4244: '=': conversion from 'ggml_float' to 'floa
t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(2202,20): warning C4244: '=': conversion from 'ggml_float' to 'floa
t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(2341,10): warning C4244: '=': conversion from 'ggml_float' to 'floa
t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(2924,14): warning C4996: 'strdup': The POSIX name for this item is
deprecated. Instead, use the ISO C and C++ conformant name: _strdup. See online help for details. [C:\Users\PC\Desktop
llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3179,66): warning C4244: 'function': conversion from 'int32_t' to '
const float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3211,66): warning C4244: 'function': conversion from 'float' to 'co
nst int8_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3218,68): warning C4244: 'function': conversion from 'float' to 'co
nst int16_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3225,68): warning C4244: 'function': conversion from 'float' to 'co
nst int32_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3232,72): warning C4244: 'function': conversion from 'float' to 'co
nst int32_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3279,24): warning C4244: 'return': conversion from 'float' to 'int3
2_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3284,49): warning C4244: 'return': conversion from 'float' to 'int3
2_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3292,12): warning C4244: 'return': conversion from 'float' to 'int3
2_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3323,54): warning C4244: 'function': conversion from 'int32_t' to '
float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3328,48): warning C4244: '=': conversion from 'int32_t' to 'float',
possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3360,51): warning C4244: 'return': conversion from 'int32_t' to 'fl
oat', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3394,49): warning C4244: '=': conversion from 'float' to 'int8_t',
possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3399,50): warning C4244: '=': conversion from 'float' to 'int16_t',
possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(3404,50): warning C4244: '=': conversion from 'float' to 'int32_t',
possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(4363,22): warning C4267: '=': conversion from 'size_t' to 'int', po
ssible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(4364,22): warning C4267: '=': conversion from 'size_t' to 'int', po
ssible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(4365,22): warning C4267: '=': conversion from 'size_t' to 'int', po
ssible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(4366,22): warning C4267: '=': conversion from 'size_t' to 'int', po
ssible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(5833,33): warning C4244: 'initializing': conversion from 'ggml_floa
t' to 'float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(5844,38): warning C4244: 'initializing': conversion from 'ggml_floa
t' to 'float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(5916,33): warning C4244: 'initializing': conversion from 'ggml_floa
t' to 'float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6011,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6013,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6014,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6015,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6018,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6020,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6021,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6022,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6024,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6025,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6026,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6027,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6174,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6175,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6176,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6177,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6179,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6180,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6181,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6182,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6184,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6185,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6186,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6187,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6386,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6387,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6388,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6389,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6391,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6392,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6393,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6394,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6396,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6397,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6398,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6399,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(6974,35): warning C4244: 'function': conversion from 'ggml_float' t
o 'const float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7031,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7032,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7033,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7034,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7088,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7089,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7090,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7091,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7178,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7179,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7180,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7183,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7184,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7189,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7298,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7299,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7300,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7303,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7304,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7309,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7446,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7447,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7448,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7451,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7452,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7457,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7566,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7567,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7568,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7571,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7572,30): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7577,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7788,36): warning C4244: 'function': conversion from 'const int' to
'float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7871,38): warning C4244: 'function': conversion from 'ggml_float' t
o 'const float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7716,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7717,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7718,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7719,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7721,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7722,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7723,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7724,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7726,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7727,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7728,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7729,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7731,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7732,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7733,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7734,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7997,36): warning C4244: 'function': conversion from 'const int' to
'float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8097,38): warning C4244: 'function': conversion from 'ggml_float' t
o 'const float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7925,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7926,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7927,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7928,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7930,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7931,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7932,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7933,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7935,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7936,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7937,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7938,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7940,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7941,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7942,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(7943,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8212,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8213,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8214,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8215,27): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8217,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8218,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8219,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8220,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8222,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8227,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8228,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8229,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8230,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8232,29): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8237,28): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8238,28): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8239,28): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(8240,28): warning C4267: 'initializing': conversion from 'size_t' t
o 'const int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(9289,72): warning C4267: 'function': conversion from 'size_t' to 'i
nt', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(9554,17): warning C4996: 'fopen': This function or variable may be
unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for detail
s. [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(9809,68): warning C4244: 'function': conversion from 'int' to 'floa
t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\ggml.c(9810,68): warning C4244: 'function': conversion from 'int' to 'floa
t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.vcxproj]
ggml.vcxproj -> C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\ggml.dir\Debug\ggml.lib
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/CMakeLists.txt
llama.cpp
C:\Users\PC\Desktop\llama.net-main\llama.cpp\llama.cpp(716,35): warning C4244: 'argument': conversion from 'double' to
'float', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\llama.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\llama.cpp(1069,1): warning C4267: 'argument': conversion from 'size_t' to
'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\llama.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\llama.cpp(1238,14): warning C4244: '/=': conversion from 'double' to 'floa
t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\llama.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\llama.cpp(1254,25): warning C4244: '*=': conversion from 'double' to '_Ty'
, possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\llama.vcxproj]
with
[
_Ty=float
]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\llama.cpp(1581,27): warning C4244: '=': conversion from 'time_t' to 'int',
possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\llama.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\llama.cpp(1706,22): warning C4267: 'return': conversion from 'size_t' to '
int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\llama.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\llama.cpp(1710,41): warning C4267: 'return': conversion from 'size_t' to '
int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\llama.vcxproj]
Creating library C:/Users/PC/Desktop/llama.net-main/llama.cpp/build/Debug/llama.lib and object C:/Users/PC/Desktop
/llama.net-main/llama.cpp/build/Debug/llama.exp
llama.vcxproj -> C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\bin\Debug\llama.dll
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/examples/CMakeLists.txt
common.cpp
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\common.cpp(262,78): warning C4267: 'argument': conversion from 's
ize_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\common.vcxproj]
common.vcxproj -> C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\common.dir\Debug\common.lib
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/examples/embedding/CMakeLists.txt
embedding.cpp
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\embedding\embedding.cpp(20,27): warning C4244: '=': conversion fr
om 'time_t' to 'int32_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\embedding
embedding.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\embedding\embedding.cpp(82,16): warning C4267: 'argument': conver
sion from 'size_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\embeddi
ng\embedding.vcxproj]
common.obj : error LNK2019: unresolved external symbol ggml_mlock_supported referenced in function "void __cdecl gpt_pr
int_usage(int,char * *,struct gpt_params const &)" (?gpt_print_usage@@YAXHPEAPEADAEBUgpt_params@@@z) [C:\Users\PC\Deskt
op\llama.net-main\llama.cpp\build\examples\embedding\embedding.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\bin\Debug\embedding.exe : fatal error LNK1120: 1 unresolved external
s [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\embedding\embedding.vcxproj]
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/examples/main/CMakeLists.txt
main.cpp
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\main\main.cpp(77,27): warning C4244: '=': conversion from 'time_t
' to 'int32_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\main\main.cpp(122,72): warning C4267: 'argument': conversion from
'size_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxpro
j]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\main\main.cpp(127,91): warning C4267: 'argument': conversion from
'size_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxpro
j]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\main\main.cpp(267,16): warning C4267: 'argument': conversion from
'size_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxpro
j]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\main\main.cpp(273,28): warning C4267: '+=': conversion from 'size
t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\main\main.cpp(405,53): warning C4267: '=': conversion from 'size
t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\main\main.cpp(417,46): warning C4267: '-=': conversion from 'size
_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxproj]
common.obj : error LNK2019: unresolved external symbol ggml_mlock_supported referenced in function "void __cdecl gpt_pr
int_usage(int,char * *,struct gpt_params const &)" (?gpt_print_usage@@YAXHPEAPEADAEBUgpt_params@@@z) [C:\Users\PC\Deskt
op\llama.net-main\llama.cpp\build\examples\main\main.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\bin\Debug\main.exe : fatal error LNK1120: 1 unresolved externals [C:
\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\main\main.vcxproj]
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/examples/perplexity/CMakeLists.txt
perplexity.cpp
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\perplexity\perplexity.cpp(18,59): warning C4244: '/=': conversion
from 'double' to '_Ty', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\perplexity
perplexity.vcxproj]
with
[
_Ty=float
]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\perplexity\perplexity.cpp(29,49): warning C4267: 'initializing':
conversion from 'size_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\p
erplexity\perplexity.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\perplexity\perplexity.cpp(41,12): warning C4267: 'argument': conv
ersion from 'size_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\perpl
exity\perplexity.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\examples\perplexity\perplexity.cpp(97,27): warning C4244: '=': conversion
from 'time_t' to 'int32_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\perplexi
ty\perplexity.vcxproj]
common.obj : error LNK2019: unresolved external symbol ggml_mlock_supported referenced in function "void __cdecl gpt_pr
int_usage(int,char * *,struct gpt_params const &)" (?gpt_print_usage@@YAXHPEAPEADAEBUgpt_params@@@z) [C:\Users\PC\Deskt
op\llama.net-main\llama.cpp\build\examples\perplexity\perplexity.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\bin\Debug\perplexity.exe : fatal error LNK1120: 1 unresolved externa
ls [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\perplexity\perplexity.vcxproj]
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/examples/quantize/CMakeLists.txt
quantize.cpp
quantize.obj : error LNK2019: unresolved external symbol ggml_time_init referenced in function main [C:\Users\PC\Deskto
p\llama.net-main\llama.cpp\build\examples\quantize\quantize.vcxproj]
quantize.obj : error LNK2019: unresolved external symbol ggml_time_us referenced in function main [C:\Users\PC\Desktop
llama.net-main\llama.cpp\build\examples\quantize\quantize.vcxproj]
quantize.obj : error LNK2019: unresolved external symbol ggml_init referenced in function main [C:\Users\PC\Desktop\lla
ma.net-main\llama.cpp\build\examples\quantize\quantize.vcxproj]
quantize.obj : error LNK2019: unresolved external symbol ggml_free referenced in function main [C:\Users\PC\Desktop\lla
ma.net-main\llama.cpp\build\examples\quantize\quantize.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\bin\Debug\quantize.exe : fatal error LNK1120: 4 unresolved externals
[C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\examples\quantize\quantize.vcxproj]
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/tests/CMakeLists.txt
test-quantize.c
C:\Users\PC\Desktop\llama.net-main\llama.cpp\tests\test-quantize.c(23,61): warning C4244: 'initializing': conversion fr
om 'float' to 'uint8_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\tests\test-quantize.
vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\tests\test-quantize.c(37,31): warning C4244: 'initializing': conversion fr
om 'float' to 'uint8_t', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\tests\test-quantize.
vcxproj]
test-quantize.obj : error LNK2019: unresolved external symbol ggml_quantize_q4_0 referenced in function main [C:\Users
PC\Desktop\llama.net-main\llama.cpp\build\tests\test-quantize.vcxproj]
test-quantize.obj : error LNK2019: unresolved external symbol ggml_quantize_q4_1 referenced in function main [C:\Users
PC\Desktop\llama.net-main\llama.cpp\build\tests\test-quantize.vcxproj]
C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\bin\Debug\test-quantize.exe : fatal error LNK1120: 2 unresolved exte
rnals [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\tests\test-quantize.vcxproj]
Building Custom Rule C:/Users/PC/Desktop/llama.net-main/llama.cpp/tests/CMakeLists.txt
test-tokenizer-0.cpp
C:\Users\PC\Desktop\llama.net-main\llama.cpp\tests\test-tokenizer-0.cpp(52,94): warning C4267: 'argument': conversion f
rom 'size_t' to 'int', possible loss of data [C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\tests\test-tokenizer-0
.vcxproj]
test-tokenizer-0.vcxproj -> C:\Users\PC\Desktop\llama.net-main\llama.cpp\build\bin\Debug\test-tokenizer-0.exe

hpretila Apr 3, 2023
Author

@afsanchez001 Apologies for not responding to this specific log. I'm not too sure why VS 22 doesn't compile it properly in this instance. Can you make an issue on my repository? I have only tested against VS19, gcc on Linux and clang on Mac.

afsanchez001 · 2023-03-31T17:25:57Z

afsanchez001
Mar 31, 2023

When I run the project I get this:

System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'

It occurs at line 26 of LLaMARunner.cs

And this is the CMD window output:

======= Testing API LLaMA.NET... ==========
llama_model_load: loading model from 'weights\7B-LoRA\ggml-model-q4_0.bin' - please wait ...
llama_model_load: failed to open 'weights\7B-LoRA\ggml-model-q4_0.bin'
llama_init_from_file: failed to load model
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 2 times:
--------------------------------
   at LLaMA.NET.Native.LLaMANativeMethods.llama_tokenize(IntPtr, System.String, Int32[], Int32, Boolean)
--------------------------------
   at LLaMA.NET.LLaMARunner.WithPrompt(System.String)
   at Program.<<Main>$>g__Test_API_Inference|0_2()
   at Program.<Main>$(System.String[])

C:\Users\PC\Desktop\llama.net-main\LLaMA.NET.Testing\bin\Debug\net7.0\LLaMA.NET.Testing.exe (process 21192) exited with code -1.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

1 reply

hpretila Apr 2, 2023
Author

Yep! The reason why it's having problems is because the llama.cpp that the project is using an older version, and I suspect there's been a lot of model changes since; hence the failure to load the model. Currently sticking to the older version for now so I don't have to worry upgrading my code on every other commit while I'm still implementing core features.

You will need to use the quantize tool from the version llama.net uses to quantise and load your own model for this specific version. I'm still working on creating a shim for that, so you will have to do it by hand. If you built it with python scripts/build_llama_cpp.py, this quantiser is on .\llama.cpp\build\bin\Debug\quantize.exe.

Refer to llama.cpp's README on how to convert and quantise your own models, but essentially follow the following steps:

# obtain the original LLaMA model weights and place them in ./models
ls ./models
65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model

# install Python dependencies
python3 -m pip install torch numpy sentencepiece

# convert the 7B model to ggml FP16 format
python3 convert-pth-to-ggml.py models/7B/ 1

# quantize the model to 4-bits (using method 2 = q4_0)
./path/to/quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2

afsanchez001 · 2023-03-31T17:26:51Z

afsanchez001
Mar 31, 2023

I have never used CMake before, so I am a total newbie. My hope is that sharing the output from the cmake command, it would let you spot my error. Thank you sir!

I am not sure that this matters, but I am using Visual Studio 2022 Community Edition, and my dev system is an AMD processor.

AMD Ryzen 9 3900X 12-Core Processor, 3.79 GHz
32.0 GB 2133MHz DIMM
NVIDIA GeForce RTX 2060
Two Samsung SSD 860 EVO M.2 1TB drives

I am not sure if the hardware matters for this project?

0 replies

afsanchez001 · 2023-04-01T18:39:24Z

afsanchez001
Apr 1, 2023

https://stackoverflow.com/questions/75908121/c-sharp-project-referencing-c-dll-getting-system-accessviolationexception-at

I opened that question for C++ help as I think the problem is with the llama.cpp build.

0 replies

dranger003 · 2023-04-02T02:36:21Z

dranger003
Apr 2, 2023

Got bored today and slapped together a .NET Core library wrapped around llama.cpp! Got inference working, but still need to work through the model conversions 😁
using LLaMA.NET;

LLaMAModel model = LLaMAModel.FromPath("/path/to/7B-LoRA/ggml-model-q4_0.bin");
LLaMARunner runner = model.CreateRunner()

var res = runner
    .WithPrompt(" This is the story of a man named ")
    .Infer(out _, nTokensToPredict = 50);

Console.Write(res);

model.Dispose();

Nice work! Looking forward to future updates, thanks for sharing. I have been playing around with using llama.cpp in dotnet on my end as well and I created a quick project for fun - if you want to check it out.

I am essentially compiling main.exe as a main.dll instead and then using DllImport then I added two hook functions (one for fetching the next input and another for the next output).

3 replies

hpretila Apr 2, 2023
Author

Woah this is neat! Thank you for sharing, it's a much more lightweight approach without the scripting I ended up using for mine.

x4080 May 16, 2023

And it's auto updated when llama cpp is updated, i wondered if there's any speed overhead using dot net?

hpretila Jun 9, 2023
Author

@x4080 to be fair you still have to rebuild llama cpp every time for all the dotnet libraries out there 😅

AsakusaRinne · 2023-06-24T04:06:12Z

AsakusaRinne
Jun 24, 2023

Hi all, I hope I could have seen this discussion earlier. We also made a repo LLamaSharp and have released NuGet packages to allow users to use it without compiling themselves. It provides many features and will be long-time maintained by SciSharp community. May I kindly ask that if you would like to develop it together with us since you all are also experienced with LLM in dotnet? Currently there are many feature requests so the amount of the work is a little big for me. I think gathering the community effort will make the LLama toolchain in dotnet better. I'll be very glad and many thanks if you'd like to make it together 😊

0 replies

neuhaus · 2023-06-24T13:15:02Z

neuhaus
Jun 24, 2023

Got bored today and slapped together a .NET Core library wrapped around llama.cpp!

Very cool! I was going to use it today but it seems to be 3 months behind llama.cpp. Care to update it?

1 reply

hpretila Jul 22, 2023
Author

Hi! The most used repository for a DotNET library is now SciSharp/LLamaSharp if you wanted to support a more active development. George has endorsed it on the repository's readme!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.NET LLaMA.cpp library! 🦙 #494

{{title}}

Replies: 7 comments 8 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

.NET LLaMA.cpp library! 🦙 #494

Replies: 7 comments · 8 replies

hpretila Mar 31, 2023 Author

hpretila Apr 3, 2023 Author

hpretila Apr 2, 2023 Author

hpretila Apr 2, 2023 Author

hpretila Jun 9, 2023 Author

hpretila Jul 22, 2023 Author

Replies: 7 comments 8 replies

hpretila Mar 31, 2023
Author

hpretila Apr 3, 2023
Author

hpretila Apr 2, 2023
Author

hpretila Apr 2, 2023
Author

hpretila Jun 9, 2023
Author

hpretila Jul 22, 2023
Author