如何通过cgo/JNA将Java字符串数组[]String传递给Go？

huangapple
未分类评论65阅读模式
英文:

How to pass a Java string array []String to Go via cgo/JNA

问题

我想通过JNA将Java中的String[]传递给我的Go函数。

我的Go函数具有以下签名：

  1. func PredicateEval(keys, values []string, expression string) *C.char

我已经使用链接模式“c-shared”编译了Go库。我在Java中定义了一个_GoString_，如下所示：

  1. package predicates;
  2. import com.ochafik.lang.jnaerator.runtime.NativeSize;
  3. import com.sun.jna.Pointer;
  4. import com.sun.jna.Structure;
  5. import java.util.Arrays;
  6. import java.util.List;
  8. public class _GoString_ extends Structure {
  9. 	public Pointer p;
  10. 	public NativeSize n;
  11. 	public _GoString_() {
  12. 		super();
  13. 	}
  14. 	protected List<String> getFieldOrder() {
  15. 		return Arrays.asList("p", "n");
  16. 	}
  17. 	public _GoString_(Pointer p, NativeSize n) {
  18. 		super();
  19. 		this.p = p;
  20. 		this.n = n;
  21. 	}
  22. 	public static class ByReference extends _GoString_ implements Structure.ByReference {
  23. 		
  24. 	};
  25. 	public static class ByValue extends _GoString_ implements Structure.ByValue {
  26. 		
  27. 	};
  28. }

我还定义了一个Go切片，如下所示：

  1. package predicates;
  2. import com.sun.jna.Pointer;
  3. import com.sun.jna.Structure;
  4. import java.util.Arrays;
  5. import java.util.List;
  7. public class GoSlice extends Structure {
  8. 	public Pointer data;
  9. 	public long len;
  10. 	public long cap;
  11. 	public GoSlice() {
  12. 		super();
  13. 	}
  14. 	protected List<String> getFieldOrder() {
  15. 		return Arrays.asList("data", "len", "cap");
  16. 	}
  17. 	public GoSlice(Pointer data, long len, long cap) {
  18. 		super();
  19. 		this.data = data;
  20. 		this.len = len;
  21. 		this.cap = cap;
  22. 	}
  23. 	public static class ByReference extends GoSlice implements Structure.ByReference {
  24. 		
  25. 	};
  26. 	public static class ByValue extends GoSlice implements Structure.ByValue {
  27. 		
  28. 	};
  29. }

这是我将Java的String[]转换为Go的string[]的尝试：

  1. static {
  2.     try {
  3.       Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
  4.       field.setAccessible(true);
  5.       unsafe = (sun.misc.Unsafe) field.get(null);
  6.       Class<?> clazz = ByteBuffer.allocateDirect(0).getClass();
  7.       DIRECT_BYTE_BUFFER_ADDRESS_OFFSET = unsafe.objectFieldOffset(Buffer.class.getDeclaredField("address"));
  8.       DIRECT_BYTE_BUFFER_CLASS = clazz;
  9.     } catch (Exception e) {
  10.       throw new AssertionError(e);
  11.     }
  12. }
  14. private static long getAddress(ByteBuffer buffer) {
  15.     assert buffer.getClass() == DIRECT_BYTE_BUFFER_CLASS;
  16.     return unsafe.getLong(buffer, DIRECT_BYTE_BUFFER_ADDRESS_OFFSET);
  17. }
  19. public static _GoString_.ByValue JavaStringToGo(String jstr) {
  20.     try {
  21.       byte[] bytes = jstr.getBytes("utf-8");
  22.       ByteBuffer bb = ByteBuffer.allocateDirect(bytes.length);
  23.       bb.put(bytes);
  24.       Pointer p = new Pointer(getAddress(bb));
  25.       _GoString_.ByValue value = new _GoString_.ByValue();
  27.       value.n = new NativeSize(bytes.length);
  28.       value.p = p;
  29.       return value;
  30.     } catch (UnsupportedEncodingException e) {
  31.       throw new RuntimeException(e);
  32.     }
  33. }
  35. public static GoSlice.ByValue JavaStringArrayToGoStringSlice(String[] strings) {
  36.     _GoString_.ByValue[] goStrings = new _GoString_.ByValue[strings.length];
  37.     for (int i = 0; i < strings.length; i++) {
  38.       goStrings[i] = JavaStringToGo(strings[i]);
  39.     }
  41.     Memory arr = new Memory(strings.length * Native.getNativeSize(_GoString_.class));
  42.     for (int i = 0; i < strings.length; i++) {
  43.       byte[] bytes = goStrings[0].getPointer().getByteArray(0, Native.getNativeSize(_GoString_.class));
  44.       arr.write(i*Native.getNativeSize(_GoString_.class), bytes, 0, bytes.length);
  45.     }
  46.     GoSlice.ByValue slice = new GoSlice.ByValue();
  47.     slice.data = arr;
  48.     slice.len = strings.length;
  49.     slice.cap = strings.length;
  51.     return slice;
  52. }

一切都编译通过，但是当我尝试在Go端访问切片元素时，我得到了一个段错误：

  1. unexpected fault address 0xb01dfacedebac1e
  2. fatal error: fault
  3. [signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x10d7d3d6f]
  5. goroutine 17 [running, locked to thread]:
英文:

I would like to pass a String[] in Java to my Go function via JNA.

My go function has the following signature:

  1. func PredicateEval(keys, values []string, expression string) *C.char

I have compiled the go library with link mode as "c-shared". I have a GoString in java defined as:

  1. package predicates;
  2. import com.ochafik.lang.jnaerator.runtime.NativeSize;
  3. import com.sun.jna.Pointer;
  4. import com.sun.jna.Structure;
  5. import java.util.Arrays;
  6. import java.util.List;
  7. /**
  8.  * &lt;i&gt;native declaration : coverage_server/predicate_jvm_bridge/lib/libtest.h&lt;/i&gt;&lt;br&gt;
  9.  * This file was autogenerated by &lt;a href=&quot;http://jnaerator.googlecode.com/&quot;&gt;JNAerator&lt;/a&gt;,&lt;br&gt;
  10.  * a tool written by &lt;a href=&quot;http://ochafik.com/&quot;&gt;Olivier Chafik&lt;/a&gt; that &lt;a href=&quot;http://code.google.com/p/jnaerator/wiki/CreditsAndLicense&quot;&gt;uses a few opensource projects.&lt;/a&gt;.&lt;br&gt;
  11.  * For help, please visit &lt;a href=&quot;http://nativelibs4java.googlecode.com/&quot;&gt;NativeLibs4Java&lt;/a&gt; , &lt;a href=&quot;http://rococoa.dev.java.net/&quot;&gt;Rococoa&lt;/a&gt;, or &lt;a href=&quot;http://jna.dev.java.net/&quot;&gt;JNA&lt;/a&gt;.
  12.  */
  13. public class _GoString_ extends Structure {
  14. 	/** C type : const char* */
  15. 	public Pointer p;
  16. 	public NativeSize n;
  17. 	public _GoString_() {
  18. 		super();
  19. 	}
  20. 	protected List&lt;String&gt; getFieldOrder() {
  21. 		return Arrays.asList(&quot;p&quot;, &quot;n&quot;);
  22. 	}
  23. 	/** @param p C type : const char* */
  24. 	public _GoString_(Pointer p, NativeSize n) {
  25. 		super();
  26. 		this.p = p;
  27. 		this.n = n;
  28. 	}
  29. 	public static class ByReference extends _GoString_ implements Structure.ByReference {
  30. 		
  31. 	};
  32. 	public static class ByValue extends _GoString_ implements Structure.ByValue {
  33. 		
  34. 	};
  35. }

I also have a Go slice defined as:

  1. package predicates;
  2. import com.sun.jna.Pointer;
  3. import com.sun.jna.Structure;
  4. import java.util.Arrays;
  5. import java.util.List;
  6. /**
  7.  * &lt;i&gt;native declaration : coverage_server/predicate_jvm_bridge/lib/libtest.h&lt;/i&gt;&lt;br&gt;
  8.  * This file was autogenerated by &lt;a href=&quot;http://jnaerator.googlecode.com/&quot;&gt;JNAerator&lt;/a&gt;,&lt;br&gt;
  9.  * a tool written by &lt;a href=&quot;http://ochafik.com/&quot;&gt;Olivier Chafik&lt;/a&gt; that &lt;a href=&quot;http://code.google.com/p/jnaerator/wiki/CreditsAndLicense&quot;&gt;uses a few opensource projects.&lt;/a&gt;.&lt;br&gt;
  10.  * For help, please visit &lt;a href=&quot;http://nativelibs4java.googlecode.com/&quot;&gt;NativeLibs4Java&lt;/a&gt; , &lt;a href=&quot;http://rococoa.dev.java.net/&quot;&gt;Rococoa&lt;/a&gt;, or &lt;a href=&quot;http://jna.dev.java.net/&quot;&gt;JNA&lt;/a&gt;.
  11.  */
  12. public class GoSlice extends Structure {
  13. 	/** C type : void* */
  14. 	public Pointer data;
  15. 	/** C type : GoInt */
  16. 	public long len;
  17. 	/** C type : GoInt */
  18. 	public long cap;
  19. 	public GoSlice() {
  20. 		super();
  21. 	}
  22. 	protected List&lt;String&gt; getFieldOrder() {
  23. 		return Arrays.asList(&quot;data&quot;, &quot;len&quot;, &quot;cap&quot;);
  24. 	}
  25. 	/**
  26. 	 * @param data C type : void*&lt;br&gt;
  27. 	 * @param len C type : GoInt&lt;br&gt;
  28. 	 * @param cap C type : GoInt
  29. 	 */
  30. 	public GoSlice(Pointer data, long len, long cap) {
  31. 		super();
  32. 		this.data = data;
  33. 		this.len = len;
  34. 		this.cap = cap;
  35. 	}
  36. 	public static class ByReference extends GoSlice implements Structure.ByReference {
  37. 		
  38. 	};
  39. 	public static class ByValue extends GoSlice implements Structure.ByValue {
  40. 		
  41. 	};
  42. }

This is my attempt to convert a Java []String to a Go string[].

  1.   static {
  2.     try {
  3.       Field field = sun.misc.Unsafe.class.getDeclaredField(&quot;theUnsafe&quot;);
  4.       field.setAccessible(true);
  5.       unsafe = (sun.misc.Unsafe) field.get(null);
  6.       Class&lt;?&gt; clazz = ByteBuffer.allocateDirect(0).getClass();
  7.       DIRECT_BYTE_BUFFER_ADDRESS_OFFSET = unsafe.objectFieldOffset(Buffer.class.getDeclaredField(&quot;address&quot;));
  8.       DIRECT_BYTE_BUFFER_CLASS = clazz;
  9.     } catch (Exception e) {
  10.       throw new AssertionError(e);
  11.     }
  12.   }
  14.   private static long getAddress(ByteBuffer buffer) {
  15.     assert buffer.getClass() == DIRECT_BYTE_BUFFER_CLASS;
  16.     return unsafe.getLong(buffer, DIRECT_BYTE_BUFFER_ADDRESS_OFFSET);
  17.   }
  19.   public static _GoString_.ByValue JavaStringToGo(String jstr) {
  20.     try {
  21.       byte[] bytes = jstr.getBytes(&quot;utf-8&quot;);
  22.       //ByteBuffer bb = ByteBuffer.wrap(bytes);
  23.       ByteBuffer bb = ByteBuffer.allocateDirect(bytes.length);
  24.       bb.put(bytes);
  25.       Pointer p = new Pointer(getAddress(bb));
  26.       _GoString_.ByValue value = new _GoString_.ByValue();
  28.       value.n = new NativeSize(bytes.length);
  29.       value.p = p;
  30.       return value;
  31.     } catch (UnsupportedEncodingException e) {
  32.       throw new RuntimeException(e);
  33.     }
  34.   }
  35.   
  36.   public static GoSlice.ByValue JavaStringArrayToGoStringSlice(String[] strings) {
  37.     _GoString_.ByValue[] goStrings = new _GoString_.ByValue[strings.length];
  38.     for (int i = 0; i &lt; strings.length; i++) {
  39.       goStrings[i] = JavaStringToGo(strings[i]);
  40.     }
  42.     Memory arr = new Memory(strings.length * Native.getNativeSize(_GoString_.class));
  43.     for (int i = 0; i &lt; strings.length; i++) {
  44.       System.out.println(Native.getNativeSize(_GoString_.class));
  45.       byte[] bytes = goStrings[0].getPointer().getByteArray(0, Native.getNativeSize(_GoString_.class));
  46.       arr.write(i*Native.getNativeSize(_GoString_.class), bytes, 0, bytes.length);
  47.     }
  48.     GoSlice.ByValue slice = new GoSlice.ByValue();
  49.     slice.data = arr;
  50.     slice.len = strings.length;
  51.     slice.cap = strings.length;
  53.     return slice;
  54.   }

Everything compiles, but when i try to access the slice elements on the Go side, i get a seg fault:

  1. unexpected fault address 0xb01dfacedebac1e
  2. fatal error: fault
  3. [signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x10d7d3d6f]
  5. goroutine 17 [running, locked to thread]:

答案1

得分: 0

您正在失去对实际字符串内存分配的追踪（和控制）。

您的 _GoString_ 的映射只包括指针的分配（4 或 8 字节）和 NativeSize（一个 4 或 8 字节的 size_t）。这个映射假设 Pointer 保持有效：

  1. public class _GoString_ extends Structure {
  2.     /** C type : const char* */
  3.     public Pointer p;
  4.     public NativeSize n;
  5.     // constructors, etc.
  6. }

然而，当您将值分配给 p 时，您只跟踪指针的地址，而不是实际的内存分配（我在您的代码中添加了注释）：

  1.   public static _GoString_.ByValue JavaStringToGo(String jstr) {
  2.     try {
  3.       byte[] bytes = jstr.getBytes("utf-8");
  4.       // 
  5.       // 在这里，您为字节数组分配内存
  6.       //
  7.       ByteBuffer bb = ByteBuffer.allocateDirect(bytes.length);
  8.       bb.put(bytes);
  9.       // 
  10.       // 在这里，您只跟踪字节数组的指针
  11.       //
  12.       Pointer p = new Pointer(getAddress(bb));
  13.       // 
  14.       // 您再也没有引用 bb，它不再可达
  15.       // 系统可以回收它的分配
  16.       //
  17.       _GoString_.ByValue value = new _GoString_.ByValue();
  19.       value.n = new NativeSize(bytes.length);
  20.       value.p = p;
  21.       return value;
  22.     } catch (UnsupportedEncodingException e) {
  23.       throw new RuntimeException(e);
  24.     }
  25.   }

作为直接的字节缓冲区，内存位置和（解）分配机制与普通对象和 GC 不同，但基本原则仍然适用，即一旦您失去对 Java 对象（ByteBuffer）的引用，就无法控制本机内存何时被释放。（当 bb 被 GC 时，它的内部字段包括将在处理时触发释放的引用。）

一种可能的解决方案是在您的 _GoString_ 类中添加一个 private 字段，该字段保持对 ByteBuffer 的强引用，并防止系统回收其内存（可能需要添加一个 ByteBuffer 构造函数）。

另一种解决方案是对字符串使用 JNA 的 Memory 类，并直接将该 Memory 对象（它扩展了 Pointer）存储到 p 字段中。我不确定为什么您选择了直接字节缓冲区用于此应用程序，所以这可能不适用于您的用例，但它肯定会简化您的代码。

英文:

You're losing track (and control) of the memory allocation of the actual Strings.

Your mapping of _GoString_ only includes allocation of a Pointer (4 or 8 bytes) and a NativeSize (a 4- or 8-byte size_t). This mapping assumes the Pointer remains valid:

  1. public class _GoString_ extends Structure {
  2.     /** C type : const char* */
  3.     public Pointer p;
  4.     public NativeSize n;
  5.     // constructors, etc.
  6. }

However, when you assign the value to p you only keep track of the pointer's address and not the actual memory allocation (I've added comments to your code):

  1.   public static _GoString_.ByValue JavaStringToGo(String jstr) {
  2.     try {
  3.       byte[] bytes = jstr.getBytes(&quot;utf-8&quot;);
  4.       // 
  5.       // Here you allocate memory for the bytes
  6.       //
  7.       ByteBuffer bb = ByteBuffer.allocateDirect(bytes.length);
  8.       bb.put(bytes);
  9.       // 
  10.       // Here you only keep track of the pointer to the bytes
  11.       //
  12.       Pointer p = new Pointer(getAddress(bb));
  13.       // 
  14.       // You never reference bb again, it is no longer reachable
  15.       // and its allocation can be reclaimed by the system
  16.       //
  17.       _GoString_.ByValue value = new _GoString_.ByValue();
  19.       value.n = new NativeSize(bytes.length);
  20.       value.p = p;
  21.       return value;
  22.     } catch (UnsupportedEncodingException e) {
  23.       throw new RuntimeException(e);
  24.     }
  25.   }

As a direct bytebuffer, the memory location and (de)allocation mechanism are different than normal objects and GC, but the fundamental principle applies that once you've lost reference to the Java object (the ByteBuffer) you have no control over when that native memory will be freed. (When bb is GC'd, its internal fields include the reference that will trigger the deallocation when it is processed.)

One possible solution is to add a private field to your _GoString_ class that maintains a strong reference to the ByteBuffer and prevents the system from reclaiming its memory (possibly adding a ByteBuffer constrcutor).

Another solution is to use JNA's Memory class for the Strings and directly store that Memory object (which extends Pointer) to the p field. I'm not sure why you have chosen direct byte buffers for this application, so this may not apply to your use case but it would certainly simplify your code.

huangapple
  • 本文由 发表于 2021年7月21日 15:23:09
  • 转载请务必保留本文链接：https://java.coder-hub.com/68465480.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定