First pass at maven refactoring #2
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 7 commits
January 13, 2013 16:22
thallgren
added a commit
that referenced
this pull request
Jan 14, 2013
First pass at maven refactoring
|
This looks good to me. There's one thing that hasn't been discussed and that's how to handle the Eclipse artifacts (.settings, .classpath, and .project). I've found that the .settings in particular can be very useful when it's desirable to enforce coding format and validation checks. Requires that the Eclipse IDE is used of course. Then again, if it isn't, then those artifacts will cause no harm. But we can revisit that later. |
|
Ah, yes. Throughout the years I've found it far easier to enforce such in the build, rather than relying on the proactive use in the IDE. That way conventions are enforced uniformly. Maven has some nice plugins for such. Would that be acceptable? If not, it's easy enough to back out the ignores and check on the settings. Sent from my Tricorder On Jan 14, 2013, at 1:14 AM, Thomas Hallgren notifications@github.com wrote:
|
beargiles
added a commit
to beargiles/pljava
that referenced
this pull request
May 28, 2025
This is prototype based on the blackhole_fdw. That is - the required method are the minimum required for the FDW to be loaded. There's quite a bit of extra code required if before you can use a real backend that's just happens to also return nothing. I have already implemented that elsewhere but for now it's fine to skip it. That said most of the provided methods call the appropriate java method. THESE ARE NOT STATIC METHODS - it is assumed that everything except for the FDWValidator is associated with a Java object - FDWForeignDataWrapper, FDWServer, FDWTable, or the various internal states. The `fwd_private` fields contain a `jobject` for the java `instance`. It is passed through to java via the JNI call in the same way that we can use `method.invoke(Object obj,...)` in Java. There is also a dummy implementation of the java side of the FDW. It does nothing but log an acknowledgement that the method was called. (Is there a way to call trigger `elog(NOTIFY,...)` ? BONUS - and probably a separate pull request The pljava-so directory contains a Dockerfile and docker-compose.yml file that can be used to create a test image. The makefile(?) needs to add a stanza that calls `docker build ...` - I can provide the details later. For this project it may make more sense to move this to the packaging module - for the other project I'm working on it's entirely standalone so I can do everything in that module. Once there's a docker image I can add an 'example' that uses TestContainers to run some actual java-based tests. KNOWN ISSUES There are a lot. The most basic is that I'm not reusing existing bits of JNI. This should be easy to fix. The most important, and the reason I haven't tried to actually build and run the .so, is that most of these objects need to stay in the persistent memory context since they live beyond the lifetime of the query. I know there's already one in the existing backend code but I don't know if I should use it or a per-FDW one. It's also clear that there needs to be an OID associated with the Foreign Data Wrapper, Server, and Foreign Table objects since they're persistence database objects. It should be possible to reuse existing ones, or at least have a clean way to access them. For now I'm using a simple tree - actually a one-to-one-to-one tree - but while the hooks are there it's not actually executing all of the steps yet. However there is enough that we can create a standalone object for testing. However this has raised a second issue - the method signatures to create a new Server or Foreign Table are identical but both Foreign Data Wrappers and Servers can support multiple children. I'm sure there's an easy way to get the jclass from a jobject but I wanted to focus on other things since we can always fall back to assuming a single matching class. There also needs to be correct mapping between int and jint, char * and jstring, etc. Once we have a solid foundation we can start adding conversions for the rest of the functions/methods. Finally the user OID is available to us but I don't know how to retrieve the desired information from it. For maximum flexibility (and security) we want at least three things: - the database username - the authentication method used - the connection method used (via unix named file, TCP/IP (IP address), etc.) The last item is important in highly secured environments, e.g., some critical operations may be limited to local users - and then the java class may still require additional authentication with something like a Yubico key. (Can you tell my other project is related to encryption keys and signing using harded external devices?) So, overall, this is enough to give a decent representation of what's required to have the C-based FDW call java classes for the actual work. There's quite a bit more work to be done on the FDW side but it's internal bookkeeping and doesn't affect the java SPI. However I don't know what prep work needs to be done beyond what's already done for UDF and UDT, and there's definitely the question of how the different persistent nodes are created and used. I've looked at a few other implementations but this goes a few steps beyond them due to the desire to eventually support multiple servers and tables. FDW Handler?... I've been a bit confused about a single handler when there has always been a desire to provide multiple servers and foreign tables. I think I have an answer though - the two functions are persistent and will have nodes and OIDs associated with them. The initial definition can point to a 'blackhole' Handler but it the handler could be replaced at the Server and Foreign table level. That's the only thing that makes sense since different tables will require different implementations of the planner, scanner, modification methods, etc. It's likely that the Validator can also change since the meaningful values for a Foreign Table may depend on the the specific Server used. BONUS ITEM tada#2 This is WAAAY out there but while digging through the deeply nested structures I came across a function pointer for performance analysis. I don't think it's limited to just FDW - it was pretty deep in the standard structures at this point. On one hand I shudder to think of the impact of proving a JNI binding to allow a java class to collect performance metrics. On the other hand.... That said... I've looked at similar situations in the past and have largely concluded that the best solution - on a single-node system - is to use a named pipe, or IPC if you're more comfortable with it. As long as you're willing to accept data loss if the system is overwhelmed there's not much risk to slamming information into either pseudodevice and relying on a consumer to move the information somewhere else. For instance something like ApacheMQ so that the analysis can be performed on one or more remote machines. However it still got me wondering... and I'm sure there's something similar for row and column-level authorization, auditing, perhaps even encryption.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the first pass at refactoring the build to maven. I tried to keep this as clean, performing a build structure refactoring with no source changes or reformatting.